Oct 30 10:42:19.153769 ip-10-0-140-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 10:42:19.153780 ip-10-0-140-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 10:42:19.153788 ip-10-0-140-230 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 10:42:19.154005 ip-10-0-140-230 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 10:42:29.403115 ip-10-0-140-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 10:42:29.403131 ip-10-0-140-230 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e48a2206ad524a0c8fced4ea2536e587 --
Oct 30 10:44:21.189266 ip-10-0-140-230 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 10:44:21.569272 ip-10-0-140-230 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 10:44:21.569272 ip-10-0-140-230 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 10:44:21.569272 ip-10-0-140-230 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 10:44:21.569272 ip-10-0-140-230 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 10:44:21.572608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572549    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.572608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572608    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572614    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572618    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572621    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572625    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572628    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572631    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572633    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572636    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572638    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572641    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572644    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572647    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572649    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572652    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572655    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572657    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572660    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572662    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.572677 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572665    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572668    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572670    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572673    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572675    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572678    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572680    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572687    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572690    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572693    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572695    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572698    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572701    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572704    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572706    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572709    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572712    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572714    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572717    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572719    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.573141 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572722    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572724    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572727    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572730    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572732    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572735    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572737    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572740    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572742    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572745    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572747    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572750    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572752    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572755    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572758    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572761    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572764    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572767    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572770    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572772    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.573637 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572776    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572779    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572781    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572784    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572787    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572789    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572792    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572794    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572797    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572799    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572802    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572804    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572808    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572813    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572816    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572819    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572822    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572825    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572828    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572831    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.574118 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572833    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572836    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572838    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572841    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572844    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572848    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572853    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572856    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572859    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572861    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572864    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572868    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572871    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572874    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572876    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572879    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572881    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572884    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572887    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.574713 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572889    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572892    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572895    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572897    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572907    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572910    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.572913    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573661    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573668    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573670    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573673    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573676    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573679    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573682    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573684    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573687    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573690    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573692    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573695    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.575506 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573699    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573701    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573704    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573707    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573710    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573713    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573716    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573718    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573722    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573725    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573727    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573730    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573732    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573735    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573738    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573740    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573745    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573750    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573753    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573756    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.576077 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573759    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573762    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573764    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573767    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573770    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573773    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573775    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573778    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573781    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573783    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573786    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573789    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573791    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573794    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573797    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573799    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573802    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573805    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573807    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573810    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.576562 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573812    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573815    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573818    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573822    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573826    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573829    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573832    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573835    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573838    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573840    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573843    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573846    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573848    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573851    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573853    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573856    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573859    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573862    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573864    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.577060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573867    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573869    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573872    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573874    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573877    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573879    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573882    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573885    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573887    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573890    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573892    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573895    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573897    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573900    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573902    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573906    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573908    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573911    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573914    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573916    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.577552 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573919    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573922    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573925    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573928    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573931    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573933    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573936    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573938    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573941    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573944    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573946    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573949    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573952    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573954    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.573957    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575212    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575228    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575234    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575239    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.578032 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575250    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575254    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575259    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575263    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575267    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575271    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575276    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575280    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575293    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575297    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575301    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575306    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575310    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575320    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575324    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575329    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575333    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575337    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575343    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.578483 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575347    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575351    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575356    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575360    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575364    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575369    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575377    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575381    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575385    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575389    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575393    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575400    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575406    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575411    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575416    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575420    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575425    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575429    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575433    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.578941 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575442    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575447    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575451    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575456    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575461    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575465    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575469    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575474    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575478    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575482    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575486    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575490    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575499    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575503    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575507    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575514    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575518    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575522    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575527    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575531    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.579409 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575535    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575540    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575544    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575548    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575552    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575560    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575564    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575568    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575573    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575577    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575581    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575585    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575589    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575593    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575597    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575601    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575608    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575619    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575624    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.579887 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575628    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575634    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575640    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575645    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575649    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575653    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575657    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575661    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575666    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575671    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575679    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575683    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575687    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575692    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575696    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575714    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575745    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575752    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575762    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575767    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.580361 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575770    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575774    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575818    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575835    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.575940    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577039    2584 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577050    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577057    2584 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577062    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577067    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577071    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577076    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577080    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577084    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577087    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577090    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577094    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577098    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577101    2584 flags.go:64] FLAG: --cgroup-root=""
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577104    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577107    2584 flags.go:64] FLAG: --client-ca-file=""
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577110    2584 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577113    2584 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 10:44:21.580821 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577119    2584 flags.go:64] FLAG: --cluster-domain=""
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577122    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577125    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577128    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577132    2584 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577136    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577139    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577142    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577145    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577148    2584 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577151    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577155    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577158    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577160    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577165    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577168    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577171    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577174    2584 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577177    2584 flags.go:64] FLAG: --enable-server="true"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577180    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577185    2584 flags.go:64] FLAG: --event-burst="100"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577188    2584 flags.go:64] FLAG: --event-qps="50"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577191    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577194    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 10:44:21.581379 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577197    2584 flags.go:64] FLAG: --eviction-hard=""
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577201    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577204    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577207    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577210    2584 flags.go:64] FLAG: --eviction-soft=""
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577213    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577216    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577219    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577222    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577225    2584 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577228    2584 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577231    2584 flags.go:64] FLAG: --feature-gates=""
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577236    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577239    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577242    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577245    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577248    2584 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577252    2584 flags.go:64] FLAG: --help="false"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577255    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577258    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577261    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577264    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577268    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 10:44:21.581987 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577272    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577275    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577278    2584 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577281    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577284    2584 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577287    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577290    2584 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577293    2584 flags.go:64] FLAG: --kube-reserved=""
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577296    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577299    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577302    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577305    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577307    2584 flags.go:64] FLAG: --lock-file=""
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577310    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577313    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577316    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577322    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577325    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577328    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577331    2584 flags.go:64] FLAG: --logging-format="text"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577333    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577337    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577340    2584 flags.go:64] FLAG: --manifest-url=""
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577343    2584 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577348    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 10:44:21.582548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577351    2584 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577354    2584 flags.go:64] FLAG: --max-pods="110"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577358    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577374    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577378    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577382    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577385    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577389    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577392    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577399    2584 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577402    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577405    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577408    2584 flags.go:64] FLAG: --pod-cidr=""
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577411    2584 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577414    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577417    2584 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577420    2584 flags.go:64] FLAG: --port="10250"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577423    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577426    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04876cbcfe57cb72c"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577430    2584 flags.go:64] FLAG: --qos-reserved=""
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577433    2584 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577436    2584 flags.go:64] FLAG: --register-node="true"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577438    2584 flags.go:64] FLAG: --register-with-taints=""
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577442    2584 flags.go:64] FLAG: --registry-burst="10"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577445    2584 flags.go:64] FLAG: --registry-qps="5"
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577448    2584 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 10:44:21.583215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577451    2584 flags.go:64] FLAG: --reserved-memory=""
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577455    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577458    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577461    2584 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577464    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577468    2584 flags.go:64] FLAG: --runonce="false"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577471    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577476    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577479    2584 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577482    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577485    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577488    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577491    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577494    2584 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577497    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577500    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577503    2584 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577506    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577509    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577512    2584 flags.go:64] FLAG: --system-cgroups=""
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577515    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577521    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577524    2584 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577527    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577531    2584 flags.go:64] FLAG: --tls-min-version=""
Oct 30 10:44:21.583820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577534    2584 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577539    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577542    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577545    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577548    2584 flags.go:64] FLAG: --v="2"
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577558    2584 flags.go:64] FLAG: --version="false"
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577562    2584 flags.go:64] FLAG: --vmodule=""
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577566    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.577570    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577702    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577706    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577710    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577713    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577715    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577720    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577723    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577726    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577728    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577731    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577734    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577737    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577740    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.584418 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577742    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577745    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577747    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577750    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577752    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577755    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577758    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577761    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577764    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577767    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577769    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577773    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577777    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577780    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577782    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577785    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577788    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577790    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577793    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.584926 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577796    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577799    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577801    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577804    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577807    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577811    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577813    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577816    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577819    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577821    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577824    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577826    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577829    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577832    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577834    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577837    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577840    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577842    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577845    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577847    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.585384 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577850    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577853    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577855    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577858    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577862    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577865    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577868    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577870    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577873    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577875    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577878    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577881    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577883    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577886    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577889    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577892    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577895    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577899    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577901    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.585857 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577904    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577907    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577910    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577912    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577915    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577918    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577920    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577923    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577926    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577928    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577930    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577933    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577936    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577939    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577942    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577944    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577947    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577951    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577954    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577957    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.586440 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577960    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577962    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577965    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577968    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577972    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577976    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577979    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577982    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577985    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577987    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577991    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577995    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.577999    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578002    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578006    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.586910 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.578045    2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578086    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578091    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578095    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578098    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578101    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578104    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578107    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578110    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578112    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578115    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578118    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578121    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578124    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578127    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578130    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578133    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578136    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578140    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.587284 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578142    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578145    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578148    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578150    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578153    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578155    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578158    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578161    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578163    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578167    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578170    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578172    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578175    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578178    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578180    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578183    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578186    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578188    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578191    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578194    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.587737 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578196    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578199    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578202    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578204    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578207    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578210    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578212    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578216    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578220    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578224    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578227    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578230    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578233    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578236    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578238    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578242    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578245    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578248    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578251    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.588228 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578254    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578256    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578260    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578263    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578266    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578268    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578271    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578274    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578276    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578279    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578282    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578284    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578287    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578290    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578292    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578295    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578298    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578300    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578303    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.588676 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578306    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578310    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578313    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578316    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578318    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578321    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578324    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578329    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578332    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578335    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578338    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578340    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578343    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578346    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578348    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578352    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578355    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578357    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578360    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578363    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.589140 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578365    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578368    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578370    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578373    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578376    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578379    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578381    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578384    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578387    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.578390    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.584858    2584 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.584872    2584 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.584888    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.584892    2584 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584955    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584961    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584964    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584968    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584973    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.589608 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584977    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584980    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584983    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584986    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584989    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584992    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584995    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.584998    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585000    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585003    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585006    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585009    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585011    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585014    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585017    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585033    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585037    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585040    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585043    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.590060 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585046    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585049    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585052    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585054    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585057    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585060    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585063    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585065    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585068    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585071    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585073    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585076    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585079    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585081    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585085    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585087    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585090    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585092    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585095    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585098    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.590502 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585101    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585103    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585106    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585108    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585111    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585113    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585116    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585119    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585121    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585124    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585126    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585130    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585132    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585135    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585138    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585140    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585143    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585146    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585149    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.590968 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585152    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585154    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585157    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585160    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585163    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585165    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585168    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585171    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585174    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585177    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585179    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585182    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585185    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585188    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585190    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585193    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585197    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585200    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585203    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585206    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.591417 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585209    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585212    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585214    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585217    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585220    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585223    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585226    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585228    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585231    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585233    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585236    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585239    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585242    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585244    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585247    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585250    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585252    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585255    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585258    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585260    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.591914 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585263    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585266    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585268    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.585296    2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585428    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585433    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585436    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585439    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585442    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585444    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585447    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585449    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585452    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585454    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585457    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585459    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.592427 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585462    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585465    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585467    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585470    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585473    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585475    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585478    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585481    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585483    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585486    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585490    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585493    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585496    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585498    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585501    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585503    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585506    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585508    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585512    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.592827 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585515    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585518    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585520    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585522    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585525    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585528    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585530    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585533    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585535    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585538    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585540    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585543    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585545    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585548    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585551    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585553    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585556    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585559    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585561    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585564    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.593293 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585567    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585569    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585572    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585574    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585577    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585579    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585582    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585585    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585589    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585593    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585595    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585598    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585601    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585604    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585606    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585609    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585612    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585614    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585617    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.593759 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585619    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585622    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585624    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585627    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585629    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585632    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585635    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585637    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585640    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585643    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585646    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585649    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585652    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585654    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585657    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585659    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585662    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585664    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585667    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585669    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.594214 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585672    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585675    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585678    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585680    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585683    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585685    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585688    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585691    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585693    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585696    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585698    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585701    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585703    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585706    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585709    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585711    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.594668 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.585734    2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585766    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585770    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585773    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585776    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585779    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585782    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585785    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585788    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585791    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585794    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585796    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585799    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585802    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585804    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585806    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585809    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585812    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585814    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:44:21.595098 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585817    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585819    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585822    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585824    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585828    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585830    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585833    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585835    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585838    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585840    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585843    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585845    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585848    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585853    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585856    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585858    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585861    2584 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585864    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585869    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:44:21.595536 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585873    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585877    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585880    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585883    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585886    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585889    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585891    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585894    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585897    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585899    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585902    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585905    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585907    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585910    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585912    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585915    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585917    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585920    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585923    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585926    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:44:21.595965 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585928    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585931    2584 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585934    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585937    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585939    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585942    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585945    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585948    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585950    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585953    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585955    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585958    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585962    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585969    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585972    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585974    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585977    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585980    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585983    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:44:21.596444 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585985    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585988    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585991    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585993    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585996    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.585998    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586001    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586003    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586007    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586009    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586012    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586014    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586017    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586038    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586041    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586044    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586046    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586049    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586052    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586055    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:44:21.596927 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586058    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586061    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586063    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586066    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586068    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586071    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586075    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586077    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586080    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:21.586085    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.586133    2584 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.589426    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.590519    2584 server.go:1014] "Starting client certificate rotation"
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.590613    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.590644    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.592762    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 10:44:21.597420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.595798    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 10:44:21.608421 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.608408    2584 log.go:25] "Validated CRI v1 runtime API"
Oct 30 10:44:21.612347 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.612331    2584 log.go:25] "Validated CRI v1 image API"
Oct 30 10:44:21.614400 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.614374    2584 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 10:44:21.618845 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.618824    2584 fs.go:135] Filesystem UUIDs: map[75d3f1e6-0c83-4693-9137-379361e5670f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dae90d51-1a36-46b8-b6ba-bc0dee7f10a6:/dev/nvme0n1p4]
Oct 30 10:44:21.618931 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.618843    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 10:44:21.624613 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.624499    2584 manager.go:217] Machine: {Timestamp:2026-10-30 10:44:21.622871043 +0000 UTC m=+0.349926175 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102894 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec285afc3436fc8d0373457373205765 SystemUUID:ec285afc-3436-fc8d-0373-457373205765 BootID:e48a2206-ad52-4a0c-8fce-d4ea2536e587 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c0:c6:5a:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c0:c6:5a:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:d5:e8:6b:fc:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 10:44:21.624613 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.624605    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 10:44:21.624794 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.624722    2584 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 10:44:21.625995 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.625971    2584 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 10:44:21.626128 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.625996    2584 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 10:44:21.626171 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.626130    2584 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 10:44:21.626171 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.626138    2584 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 10:44:21.626248 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.626175    2584 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 10:44:21.626248 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.626193    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 10:44:21.627457 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.627447    2584 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 10:44:21.632119 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.632107    2584 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 10:44:21.632235 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.632225    2584 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 10:44:21.633081 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.633071    2584 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 10:44:21.633114 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.633085    2584 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 10:44:21.633114 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.633095    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 10:44:21.633114 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.633103    2584 kubelet.go:405] "Adding apiserver pod source"
Oct 30 10:44:21.633114 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.633111    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 10:44:21.633388 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.633375    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 10:44:21.635835 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.635818    2584 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 10:44:21.643105 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.643092    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 10:44:21.644072 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644059    2584 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 10:44:21.644108 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644090    2584 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 10:44:21.644240 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644230    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 10:44:21.644274 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644247    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 10:44:21.644274 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644254    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 10:44:21.644274 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644259    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 10:44:21.644274 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644265    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 10:44:21.644274 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644272    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 10:44:21.644966 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644957    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 10:44:21.644998 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644966    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 10:44:21.644998 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644974    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 10:44:21.644998 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644986    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 10:44:21.644998 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.644999    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 10:44:21.645190 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.645007    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 10:44:21.646597 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.646583    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 10:44:21.646644 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.646601    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 10:44:21.650248 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.650228    2584 server.go:1263] "Started kubelet"
Oct 30 10:44:21.650360 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.650314    2584 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 10:44:21.650416 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.650327    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 10:44:21.650416 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.650391    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 10:44:21.650801 ip-10-0-140-230 systemd[1]: Started Kubernetes Kubelet.
Oct 30 10:44:21.652403 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.652390    2584 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 10:44:21.653339 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.653319    2584 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 10:44:21.655910 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.655894    2584 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 10:44:21.659597 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.659584    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 10:44:21.660294 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.660279    2584 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 10:44:21.660385 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.660373    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:21.661066 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661048    2584 factory.go:55] Registering systemd factory
Oct 30 10:44:21.661066 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661068    2584 factory.go:223] Registration of the systemd container factory successfully
Oct 30 10:44:21.661416 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661398    2584 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 10:44:21.661488 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661423    2584 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 10:44:21.661488 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661400    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 10:44:21.661544 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661528    2584 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 10:44:21.661544 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661533    2584 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 10:44:21.661676 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661664    2584 factory.go:153] Registering CRI-O factory
Oct 30 10:44:21.661705 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661678    2584 factory.go:223] Registration of the crio container factory successfully
Oct 30 10:44:21.661735 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661725    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 10:44:21.661766 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661738    2584 factory.go:103] Registering Raw factory
Oct 30 10:44:21.661766 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.661748    2584 manager.go:1199] Started watching for new ooms in manager
Oct 30 10:44:21.662797 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.662782    2584 manager.go:319] Starting recovery of all containers
Oct 30 10:44:21.670774 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.670748    2584 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 10:44:21.673109 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.673070    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 10:44:21.674645 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.674627    2584 manager.go:324] Recovery completed
Oct 30 10:44:21.679032 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.679005    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:21.683678 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.683662    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:21.683738 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.683688    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:21.683738 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.683699    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:21.684203 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.684184    2584 cpu_manager.go:226] "Starting" policy="none"
Oct 30 10:44:21.684203 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.684201    2584 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 10:44:21.684329 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.684220    2584 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 10:44:21.684329 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.684194    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:21.685127 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.685112    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:21.685218 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.685132    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:21.685218 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.685146    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:21.686415 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.686404    2584 policy_none.go:50] "Start"
Oct 30 10:44:21.686456 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.686424    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 10:44:21.686456 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.686434    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 10:44:21.688450 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.688437    2584 policy_none.go:44] "Start"
Oct 30 10:44:21.688828 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.688811    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 10:44:21.688886 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.688873    2584 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 10:44:21.689990 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.688815    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-230.ec2.internal.18e348bef7193985  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-230.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-230.ec2.internal,},FirstTimestamp:2026-10-30 10:44:21.650200965 +0000 UTC m=+0.377256095,LastTimestamp:2026-10-30 10:44:21.650200965 +0000 UTC m=+0.377256095,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-230.ec2.internal,}"
Oct 30 10:44:21.702687 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.702661    2584 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 10:44:21.702687 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.702600    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-230.ec2.internal.18e348bef9180511  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-230.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-230.ec2.internal,},FirstTimestamp:2026-10-30 10:44:21.683676433 +0000 UTC m=+0.410731565,LastTimestamp:2026-10-30 10:44:21.683676433 +0000 UTC m=+0.410731565,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-230.ec2.internal,}"
Oct 30 10:44:21.702815 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.702687    2584 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 10:44:21.703087 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.703067    2584 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 10:44:21.704228 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.704210    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztj6b"
Oct 30 10:44:21.713598 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.713580    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztj6b"
Oct 30 10:44:21.723689 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.723664    2584 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 10:44:21.726760 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.726743    2584 manager.go:341] "Starting Device Plugin manager"
Oct 30 10:44:21.726842 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.726772    2584 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 10:44:21.726842 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.726783    2584 server.go:87] "Starting device plugin registration server"
Oct 30 10:44:21.727011 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.726998    2584 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 10:44:21.727094 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.727011    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 10:44:21.727149 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.727100    2584 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 10:44:21.727199 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.727163    2584 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 10:44:21.727199 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.727174    2584 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 10:44:21.727696 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.727641    2584 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 10:44:21.727696 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.727683    2584 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:21.796420 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.796398    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 10:44:21.798447 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.796422    2584 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 10:44:21.798447 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.796440    2584 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 10:44:21.798447 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.796474    2584 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 10:44:21.810831 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.810815    2584 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:44:21.827743 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.827710    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:21.828481 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.828467    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:21.828574 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.828494    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:21.828574 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.828509    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:21.828574 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.828533    2584 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.848207 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.848186    2584 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.848274 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.848214    2584 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-230.ec2.internal\": node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:21.885992 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.885970    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:21.897206 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.897186    2584 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"]
Oct 30 10:44:21.897260 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.897233    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:21.897964 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.897938    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:21.898059 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.897969    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:21.898059 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.897978    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:21.900137 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900124    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:21.900269 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900255    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.900307 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900284    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:21.900792 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900780    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:21.900860 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900797    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:21.900860 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900805    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:21.900860 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900812    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:21.900860 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900832    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:21.900860 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.900842    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:21.902895 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.902882    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.902941 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.902907    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:21.903586 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.903567    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:21.903647 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.903590    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:21.903647 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.903603    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:21.927549 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.927452    2584 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-230.ec2.internal\" not found" node="ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.930107 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.930092    2584 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-230.ec2.internal\" not found" node="ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.962641 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.962625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.962720 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.962648    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.962720 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:21.962664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d42823a77ed791ee517181c1c853da97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"d42823a77ed791ee517181c1c853da97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:21.986885 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:21.986865    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.063403 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.063381    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.063403 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.063331    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.063534 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.063442    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.063534 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.063471    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d42823a77ed791ee517181c1c853da97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"d42823a77ed791ee517181c1c853da97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.063534 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.063505    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d42823a77ed791ee517181c1c853da97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"d42823a77ed791ee517181c1c853da97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.063534 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.063447    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.087317 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:22.087277    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.187546 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:22.187523    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.230688 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.230670    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.233779 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.233763    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.288556 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:22.288535    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.389116 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:22.389071    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.489625 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:22.489603    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.590257 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:22.590238    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.590606 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.590264    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 10:44:22.590606 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.590376    2584 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 10:44:22.690340 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:22.690282    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:44:22.720210 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.720143    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 10:39:22 +0000 UTC" deadline="2028-06-18 01:57:45.849695935 +0000 UTC"
Oct 30 10:44:22.720310 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.720213    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14319h13m23.129490592s"
Oct 30 10:44:22.752338 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.752318    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:44:22.753496 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.753479    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:44:22.753580 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.753516    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:44:22.753580 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.753528    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:44:22.764910 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:22.764883    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356d8c2a38ca6b7521952074b996a2cc.slice/crio-054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c WatchSource:0}: Error finding container 054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c: Status 404 returned error can't find the container with id 054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c
Oct 30 10:44:22.770872 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.770856    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 10:44:22.777739 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:22.777714    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42823a77ed791ee517181c1c853da97.slice/crio-f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771 WatchSource:0}: Error finding container f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771: Status 404 returned error can't find the container with id f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771
Oct 30 10:44:22.780708 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.780689    2584 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:44:22.798926 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.798891    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" event={"ID":"d42823a77ed791ee517181c1c853da97","Type":"ContainerStarted","Data":"f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771"}
Oct 30 10:44:22.800014 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.799995    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"356d8c2a38ca6b7521952074b996a2cc","Type":"ContainerStarted","Data":"054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c"}
Oct 30 10:44:22.863418 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.863394    2584 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.885956 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.885939    2584 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 10:44:22.887449 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.887436    2584 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:44:22.915066 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:22.915046    2584 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 10:44:23.004217 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.004150    2584 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:44:23.321784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.321711    2584 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:44:23.634225 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.634160    2584 apiserver.go:52] "Watching apiserver"
Oct 30 10:44:23.660198 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.660170    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 10:44:23.663895 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.663863    2584 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 10:44:23.664266 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.664245    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-qxz95","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-4k458","openshift-multus/network-metrics-daemon-hr78s","kube-system/konnectivity-agent-q6ddt","openshift-dns/node-resolver-887wc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz","openshift-network-diagnostics/network-check-target-ccnv5","openshift-ovn-kubernetes/ovnkube-node-5sl2r","kube-system/global-pull-secret-syncer-lkmz6","openshift-cluster-node-tuning-operator/tuned-rnv69","openshift-multus/multus-2c8hg","openshift-network-operator/iptables-alerter-2kxdq","kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"]
Oct 30 10:44:23.666993 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.666961    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.670749 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.670302    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.672629 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.672584    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.674875 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.674857    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:23.674981 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.674938    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:23.677305 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.677285    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.679758 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.679739    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.682755 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.682435    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:23.683427 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.683402    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 10:44:23.683664 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.683647    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 10:44:23.684063 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.683797    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ghh8s\""
Oct 30 10:44:23.685876 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.685848    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:44:23.685977 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.685931    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 10:44:23.686040 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.685974    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9pzxh\""
Oct 30 10:44:23.686125 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.686102    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 10:44:23.687447 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.687427    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 10:44:23.687810 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.687671    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:23.687810 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.687755    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:23.688970 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.688941    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 10:44:23.689306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.689287    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 10:44:23.689390 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.689313    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-mhmcf\""
Oct 30 10:44:23.689390 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.689349    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 10:44:23.690691 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.690670    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.693394 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.693344    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.693509 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.693418    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:23.696382 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.696000    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.698664 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.698643    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.721051 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.721033    2584 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 10:44:23.721261 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.721078    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tph4w"
Oct 30 10:44:23.733058 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.733037    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tph4w"
Oct 30 10:44:23.763284 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.763263    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 10:44:23.771687 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771663    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-lib-modules\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.771802 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771698    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.771802 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771726    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.771802 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.771802 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771796    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.772020 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771821    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.772020 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771847    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-log-socket\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.772020 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-run\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.772020 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded181d7-f3c4-4324-946f-2770e8e3d0fd-hosts-file\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.772020 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.771944    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/ded181d7-f3c4-4324-946f-2770e8e3d0fd-kube-api-access-758g5\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772044    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gljzz\" (UniqueName: \"kubernetes.io/projected/f1cb922d-534b-4fe0-8930-b115b820d656-kube-api-access-gljzz\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-agent-certs\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772112    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-etc-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772137    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb883d09-b098-410d-bc1d-f68e24021d17-ovn-node-metrics-cert\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772183    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-socket-dir-parent\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772207    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-etc-tuned\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772238    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2fpx\" (UniqueName: \"kubernetes.io/projected/1906564b-67cc-491d-b1f7-2c014046a18d-kube-api-access-r2fpx\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772263    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-konnectivity-ca\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:23.772306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772302    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phrsk\" (UniqueName: \"kubernetes.io/projected/eb883d09-b098-410d-bc1d-f68e24021d17-kube-api-access-phrsk\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772325    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-kubelet\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772350    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-conf-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772375    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twjpp\" (UniqueName: \"kubernetes.io/projected/1114f0eb-6317-4004-88fa-55f60fc991a5-kube-api-access-twjpp\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772459    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlsp5\" (UniqueName: \"kubernetes.io/projected/0df37afa-77d5-4bf7-bc83-0d7636453bda-kube-api-access-qlsp5\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772532    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-host\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772558    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-env-overrides\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772622    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-netns\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772657    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7add3842-20f6-49eb-89ad-02078f7767ef-serviceca\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772698    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtrmh\" (UniqueName: \"kubernetes.io/projected/7add3842-20f6-49eb-89ad-02078f7767ef-kube-api-access-vtrmh\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772726    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-kubelet-config\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.772773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772753    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded181d7-f3c4-4324-946f-2770e8e3d0fd-tmp-dir\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772819    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-config\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-etc-kubernetes\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772882    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-socket-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772921    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-systemd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.772959    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-var-lib-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-cnibin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773059    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-multus\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-sys\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773162    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-netd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773189    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-kubernetes\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773221    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cb922d-534b-4fe0-8930-b115b820d656-iptables-alerter-script\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773258    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cb922d-534b-4fe0-8930-b115b820d656-host-slash\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773283    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-slash\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773306    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7add3842-20f6-49eb-89ad-02078f7767ef-host\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.773408 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-sys-fs\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxbzw\" (UniqueName: \"kubernetes.io/projected/984f643d-51d1-4142-b244-f8f0980a055f-kube-api-access-hxbzw\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773384    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-netns\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773523    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-ovn\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773554    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773634    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-daemon-config\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773673    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-dbus\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-tmp\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773726    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-var-lib-kubelet\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773755    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773817    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-k8s-cni-cncf-io\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773841    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-hostroot\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-multus-certs\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773892    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:23.774236 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773951    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kml79\" (UniqueName: \"kubernetes.io/projected/dc78343b-907d-4222-b27d-7705a2a0f19e-kube-api-access-kml79\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.773979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-modprobe-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774005    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-systemd-units\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774228    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-node-log\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774270    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysconfig\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774528    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-script-lib\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774579    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-system-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774630    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-registration-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774654    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-device-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774676    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-conf\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774691    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774740    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774790    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-kubelet\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.774869 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774858    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.775430 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774907    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-systemd\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.775430 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.774950    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-cnibin\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.775620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.775600    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-bin\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.775659 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.775628    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-cni-binary-copy\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.775659 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.775643    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-bin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.792286 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.792238    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 10:44:23.792286 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.792272    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 10:44:23.792804 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.792786    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 10:44:23.792903 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.792889    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 10:44:23.792959 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.792946    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vwt65\""
Oct 30 10:44:23.793010 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.792980    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 10:44:23.793089 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793015    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 10:44:23.793089 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793034    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 10:44:23.793187 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793129    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 10:44:23.793243 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793179    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 10:44:23.793243 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793224    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 10:44:23.793243 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793228    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s5t5b\""
Oct 30 10:44:23.793393 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793302    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lntjm\""
Oct 30 10:44:23.793393 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.792977    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-472bd\""
Oct 30 10:44:23.793393 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793358    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 10:44:23.793393 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793128    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 10:44:23.793593 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793407    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:44:23.793593 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793042    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 10:44:23.793593 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793474    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 10:44:23.793593 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.793476    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 10:44:23.805176 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.805129    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5fdc6\""
Oct 30 10:44:23.805289 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.805160    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 10:44:23.805289 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.805214    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 10:44:23.805289 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.805223    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xt7nj\""
Oct 30 10:44:23.838770 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.838739    2584 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:44:23.876031 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.875993    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-conf\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.876142 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876044    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.876142 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876070    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:23.876142 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876092    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-kubelet\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.876316 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876158    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-kubelet\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.876316 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876242    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-conf\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.876316 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876297    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.876450 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876330    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-systemd\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.876450 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876355    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-cnibin\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.876450 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-bin\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.876450 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876415    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-cni-binary-copy\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876450    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-bin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876471    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-systemd\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876481    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-lib-modules\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876516    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876576    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-lib-modules\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876577    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876602    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876622    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-log-socket\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.876657 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876651    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-run\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876666    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded181d7-f3c4-4324-946f-2770e8e3d0fd-hosts-file\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876681    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/ded181d7-f3c4-4324-946f-2770e8e3d0fd-kube-api-access-758g5\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876696    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gljzz\" (UniqueName: \"kubernetes.io/projected/f1cb922d-534b-4fe0-8930-b115b820d656-kube-api-access-gljzz\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876703    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876713    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-agent-certs\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876723    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876729    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-etc-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876766    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-etc-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876784    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876805    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-bin\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876814    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb883d09-b098-410d-bc1d-f68e24021d17-ovn-node-metrics-cert\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876841    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-socket-dir-parent\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876869    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-etc-tuned\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876894    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2fpx\" (UniqueName: \"kubernetes.io/projected/1906564b-67cc-491d-b1f7-2c014046a18d-kube-api-access-r2fpx\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876917    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-konnectivity-ca\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876942    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phrsk\" (UniqueName: \"kubernetes.io/projected/eb883d09-b098-410d-bc1d-f68e24021d17-kube-api-access-phrsk\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.877268 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.876973    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-kubelet\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-conf-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877025    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twjpp\" (UniqueName: \"kubernetes.io/projected/1114f0eb-6317-4004-88fa-55f60fc991a5-kube-api-access-twjpp\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877048    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlsp5\" (UniqueName: \"kubernetes.io/projected/0df37afa-77d5-4bf7-bc83-0d7636453bda-kube-api-access-qlsp5\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877075    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877124    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-host\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877149    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-env-overrides\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-netns\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877198    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7add3842-20f6-49eb-89ad-02078f7767ef-serviceca\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877225    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtrmh\" (UniqueName: \"kubernetes.io/projected/7add3842-20f6-49eb-89ad-02078f7767ef-kube-api-access-vtrmh\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877251    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-kubelet-config\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877253    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877275    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded181d7-f3c4-4324-946f-2770e8e3d0fd-tmp-dir\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877288    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-cni-binary-copy\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877300    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-config\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877306    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-cnibin\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877372    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-bin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878013 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877420    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877433    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-socket-dir-parent\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877478    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-host\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877538    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877658    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877905    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-run\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877924    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-env-overrides\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877947    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-netns\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.877977    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded181d7-f3c4-4324-946f-2770e8e3d0fd-hosts-file\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878006    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-config\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878236    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded181d7-f3c4-4324-946f-2770e8e3d0fd-tmp-dir\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878308    2584 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878317    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-log-socket\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878360    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-kubelet-config\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878380    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7add3842-20f6-49eb-89ad-02078f7767ef-serviceca\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-etc-kubernetes\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878521    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-kubelet\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-socket-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.878812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878690    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-systemd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878720    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-var-lib-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878751    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-cnibin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878768    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-etc-kubernetes\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878822    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-var-lib-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878860    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-multus\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878868    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-cnibin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878901    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-conf-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878991    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-socket-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879018    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-systemd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.878825    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-multus\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879050    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-sys\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879079    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879119    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-netd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-kubernetes\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cb922d-534b-4fe0-8930-b115b820d656-iptables-alerter-script\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879240    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cb922d-534b-4fe0-8930-b115b820d656-host-slash\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-slash\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.879682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879290    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7add3842-20f6-49eb-89ad-02078f7767ef-host\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879315    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-sys-fs\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879377    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbzw\" (UniqueName: \"kubernetes.io/projected/984f643d-51d1-4142-b244-f8f0980a055f-kube-api-access-hxbzw\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879422    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-netns\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879428    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-netd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879443    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-ovn\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879466    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879479    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-sys-fs\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879519    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-daemon-config\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879541    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-kubernetes\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-dbus\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879612    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-tmp\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-var-lib-kubelet\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879653    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879663    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.880549 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-k8s-cni-cncf-io\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879714    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-hostroot\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879724    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-sys\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-multus-certs\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879781    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-netns\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879813    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879847    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-ovn\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879894    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-dbus\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879929    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879984    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kml79\" (UniqueName: \"kubernetes.io/projected/dc78343b-907d-4222-b27d-7705a2a0f19e-kube-api-access-kml79\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880012    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-modprobe-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880050    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-systemd-units\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880077    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-node-log\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysconfig\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-script-lib\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-system-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880257    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-system-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880291    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-konnectivity-ca\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:23.881054 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.880296    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880322    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-registration-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880357    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-device-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.880396    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret podName:bc612ba8-81c6-4d10-aecf-f8efdc6fea4f nodeName:}" failed. No retries permitted until 2026-10-30 10:44:24.380373248 +0000 UTC m=+3.107428387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret") pod "global-pull-secret-syncer-lkmz6" (UID: "bc612ba8-81c6-4d10-aecf-f8efdc6fea4f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880452    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-device-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.880320    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880510    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-slash\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880517    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-registration-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.880585    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:44:24.380566243 +0000 UTC m=+3.107621380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880623    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-modprobe-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880666    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-systemd-units\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880682    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cb922d-534b-4fe0-8930-b115b820d656-host-slash\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880725    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7add3842-20f6-49eb-89ad-02078f7767ef-host\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880728    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-k8s-cni-cncf-io\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880759    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysconfig\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.880789    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-var-lib-kubelet\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.881730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.879691    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.882364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.881243    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cb922d-534b-4fe0-8930-b115b820d656-iptables-alerter-script\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.882364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.881250    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-script-lib\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.882364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.881304    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-hostroot\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.882364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.881310    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-multus-certs\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.882364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.881343    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-node-log\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.882364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.881379    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.882364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.881969    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-daemon-config\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.883193 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.883168    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-etc-tuned\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.883483 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.883461    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb883d09-b098-410d-bc1d-f68e24021d17-ovn-node-metrics-cert\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.883483 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.883480    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-agent-certs\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:23.883683 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.883535    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-tmp\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.898335 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.898277    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 10:44:23.898335 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.898304    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 10:44:23.898335 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.898318    2584 projected.go:196] Error preparing data for projected volume kube-api-access-b42qs for pod openshift-network-diagnostics/network-check-target-ccnv5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:23.898510 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:23.898374    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs podName:167b7d19-70a6-43c4-a2c2-4386a574d3c3 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:24.398356502 +0000 UTC m=+3.125411624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b42qs" (UniqueName: "kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs") pod "network-check-target-ccnv5" (UID: "167b7d19-70a6-43c4-a2c2-4386a574d3c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:23.902077 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.902050    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/ded181d7-f3c4-4324-946f-2770e8e3d0fd-kube-api-access-758g5\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.902812 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.902785    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlsp5\" (UniqueName: \"kubernetes.io/projected/0df37afa-77d5-4bf7-bc83-0d7636453bda-kube-api-access-qlsp5\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:23.904768 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.904742    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twjpp\" (UniqueName: \"kubernetes.io/projected/1114f0eb-6317-4004-88fa-55f60fc991a5-kube-api-access-twjpp\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:23.904864 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.904838    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gljzz\" (UniqueName: \"kubernetes.io/projected/f1cb922d-534b-4fe0-8930-b115b820d656-kube-api-access-gljzz\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.905306 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.905284    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2fpx\" (UniqueName: \"kubernetes.io/projected/1906564b-67cc-491d-b1f7-2c014046a18d-kube-api-access-r2fpx\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:23.905682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.905659    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtrmh\" (UniqueName: \"kubernetes.io/projected/7add3842-20f6-49eb-89ad-02078f7767ef-kube-api-access-vtrmh\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:23.906988 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.906942    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrsk\" (UniqueName: \"kubernetes.io/projected/eb883d09-b098-410d-bc1d-f68e24021d17-kube-api-access-phrsk\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:23.908349 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.908322    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kml79\" (UniqueName: \"kubernetes.io/projected/dc78343b-907d-4222-b27d-7705a2a0f19e-kube-api-access-kml79\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:23.909111 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.909088    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxbzw\" (UniqueName: \"kubernetes.io/projected/984f643d-51d1-4142-b244-f8f0980a055f-kube-api-access-hxbzw\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:23.982269 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.982243    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-887wc"
Oct 30 10:44:23.994468 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:23.994444    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:44:23.995011 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:23.994984    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded181d7_f3c4_4324_946f_2770e8e3d0fd.slice/crio-56a24866a1d635ce8c8a021cc89c4208c214acf6ec65ba4d59ac8ca3051f4a7a WatchSource:0}: Error finding container 56a24866a1d635ce8c8a021cc89c4208c214acf6ec65ba4d59ac8ca3051f4a7a: Status 404 returned error can't find the container with id 56a24866a1d635ce8c8a021cc89c4208c214acf6ec65ba4d59ac8ca3051f4a7a
Oct 30 10:44:24.005257 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.005236    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2c8hg"
Oct 30 10:44:24.007302 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.007278    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1cb922d_534b_4fe0_8930_b115b820d656.slice/crio-a4f499333b507c41f0d264032cdea35113c6b5032c791e83932ba287418a6ffe WatchSource:0}: Error finding container a4f499333b507c41f0d264032cdea35113c6b5032c791e83932ba287418a6ffe: Status 404 returned error can't find the container with id a4f499333b507c41f0d264032cdea35113c6b5032c791e83932ba287418a6ffe
Oct 30 10:44:24.010676 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.010658    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:44:24.020445 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.020419    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1114f0eb_6317_4004_88fa_55f60fc991a5.slice/crio-3893e0286e812cd52687e6ac8f6a019efa64c80badf964f45f0c9b6a304c9fbd WatchSource:0}: Error finding container 3893e0286e812cd52687e6ac8f6a019efa64c80badf964f45f0c9b6a304c9fbd: Status 404 returned error can't find the container with id 3893e0286e812cd52687e6ac8f6a019efa64c80badf964f45f0c9b6a304c9fbd
Oct 30 10:44:24.023718 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.023549    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:44:24.023718 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.023628    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7add3842_20f6_49eb_89ad_02078f7767ef.slice/crio-685fca13f89345449294586b150e424b00aab4055adb373a7e75d1cde410e851 WatchSource:0}: Error finding container 685fca13f89345449294586b150e424b00aab4055adb373a7e75d1cde410e851: Status 404 returned error can't find the container with id 685fca13f89345449294586b150e424b00aab4055adb373a7e75d1cde410e851
Oct 30 10:44:24.025901 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.025875    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:24.035197 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.035171    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984f643d_51d1_4142_b244_f8f0980a055f.slice/crio-ef9d119ab837d6ffad02fb6eb0c44f4893af7ded3f78bfbb2903d34d72c6032c WatchSource:0}: Error finding container ef9d119ab837d6ffad02fb6eb0c44f4893af7ded3f78bfbb2903d34d72c6032c: Status 404 returned error can't find the container with id ef9d119ab837d6ffad02fb6eb0c44f4893af7ded3f78bfbb2903d34d72c6032c
Oct 30 10:44:24.037178 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.037153    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfa87f2_a44f_4a2c_bbb0_f1a6668e1657.slice/crio-0486f6c8dac9a33be956b33582bcd1aceb4e726edee6f89f4d452115f4e3126e WatchSource:0}: Error finding container 0486f6c8dac9a33be956b33582bcd1aceb4e726edee6f89f4d452115f4e3126e: Status 404 returned error can't find the container with id 0486f6c8dac9a33be956b33582bcd1aceb4e726edee6f89f4d452115f4e3126e
Oct 30 10:44:24.047835 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.047812    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:44:24.053173 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.053154    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:24.060378 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.060356    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1906564b_67cc_491d_b1f7_2c014046a18d.slice/crio-733d18f39bfd6e8244bf38c845bebaa833b6cd481466af9683f040a5095e939f WatchSource:0}: Error finding container 733d18f39bfd6e8244bf38c845bebaa833b6cd481466af9683f040a5095e939f: Status 404 returned error can't find the container with id 733d18f39bfd6e8244bf38c845bebaa833b6cd481466af9683f040a5095e939f
Oct 30 10:44:24.068288 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.068262    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb883d09_b098_410d_bc1d_f68e24021d17.slice/crio-d7436000ac04ff26222ea5a272d6bf8de3d44865f7d1705a41aebffd964ae6ee WatchSource:0}: Error finding container d7436000ac04ff26222ea5a272d6bf8de3d44865f7d1705a41aebffd964ae6ee: Status 404 returned error can't find the container with id d7436000ac04ff26222ea5a272d6bf8de3d44865f7d1705a41aebffd964ae6ee
Oct 30 10:44:24.069447 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.069402    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:44:24.082981 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:24.082959    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc78343b_907d_4222_b27d_7705a2a0f19e.slice/crio-74498fd8124d764c596898c75bc2ba32a62d0762c83634f5b5295bcdcfa6b10e WatchSource:0}: Error finding container 74498fd8124d764c596898c75bc2ba32a62d0762c83634f5b5295bcdcfa6b10e: Status 404 returned error can't find the container with id 74498fd8124d764c596898c75bc2ba32a62d0762c83634f5b5295bcdcfa6b10e
Oct 30 10:44:24.384697 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.384662    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:24.384839 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.384711    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:24.384935 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.384834    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:24.384935 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.384890    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:24.385070 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.384899    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret podName:bc612ba8-81c6-4d10-aecf-f8efdc6fea4f nodeName:}" failed. No retries permitted until 2026-10-30 10:44:25.384880889 +0000 UTC m=+4.111936029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret") pod "global-pull-secret-syncer-lkmz6" (UID: "bc612ba8-81c6-4d10-aecf-f8efdc6fea4f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:24.385070 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.384966    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:44:25.384947834 +0000 UTC m=+4.112002953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:24.486370 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.486293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:24.486515 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.486470    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 10:44:24.486515 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.486498    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 10:44:24.486515 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.486512    2584 projected.go:196] Error preparing data for projected volume kube-api-access-b42qs for pod openshift-network-diagnostics/network-check-target-ccnv5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:24.486675 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:24.486572    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs podName:167b7d19-70a6-43c4-a2c2-4386a574d3c3 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:25.486552574 +0000 UTC m=+4.213607710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b42qs" (UniqueName: "kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs") pod "network-check-target-ccnv5" (UID: "167b7d19-70a6-43c4-a2c2-4386a574d3c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:24.737652 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.737237    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 10:39:24 +0000 UTC" deadline="2028-06-27 09:17:56.9836932 +0000 UTC"
Oct 30 10:44:24.737652 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.737503    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14542h33m32.246196146s"
Oct 30 10:44:24.812412 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.812378    2584 generic.go:356] "Generic (PLEG): container finished" podID="356d8c2a38ca6b7521952074b996a2cc" containerID="ef55900939de29facc26b1680919df7ebadd34de496d33d38281ffe8e127bc4d" exitCode=0
Oct 30 10:44:24.812558 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.812489    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"356d8c2a38ca6b7521952074b996a2cc","Type":"ContainerDied","Data":"ef55900939de29facc26b1680919df7ebadd34de496d33d38281ffe8e127bc4d"}
Oct 30 10:44:24.818845 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.818760    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz" event={"ID":"984f643d-51d1-4142-b244-f8f0980a055f","Type":"ContainerStarted","Data":"ef9d119ab837d6ffad02fb6eb0c44f4893af7ded3f78bfbb2903d34d72c6032c"}
Oct 30 10:44:24.825047 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.825019    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxz95" event={"ID":"7add3842-20f6-49eb-89ad-02078f7767ef","Type":"ContainerStarted","Data":"685fca13f89345449294586b150e424b00aab4055adb373a7e75d1cde410e851"}
Oct 30 10:44:24.835225 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.835198    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2c8hg" event={"ID":"1114f0eb-6317-4004-88fa-55f60fc991a5","Type":"ContainerStarted","Data":"3893e0286e812cd52687e6ac8f6a019efa64c80badf964f45f0c9b6a304c9fbd"}
Oct 30 10:44:24.838452 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.838419    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2kxdq" event={"ID":"f1cb922d-534b-4fe0-8930-b115b820d656","Type":"ContainerStarted","Data":"a4f499333b507c41f0d264032cdea35113c6b5032c791e83932ba287418a6ffe"}
Oct 30 10:44:24.842411 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.842392    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-887wc" event={"ID":"ded181d7-f3c4-4324-946f-2770e8e3d0fd","Type":"ContainerStarted","Data":"56a24866a1d635ce8c8a021cc89c4208c214acf6ec65ba4d59ac8ca3051f4a7a"}
Oct 30 10:44:24.852773 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.852747    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" event={"ID":"d42823a77ed791ee517181c1c853da97","Type":"ContainerStarted","Data":"38d67bd062a057c0b7ebcd0ab12ed4bee5fff785966d50fdf5f0e5bac722036d"}
Oct 30 10:44:24.864012 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.863987    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerStarted","Data":"74498fd8124d764c596898c75bc2ba32a62d0762c83634f5b5295bcdcfa6b10e"}
Oct 30 10:44:24.867792 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.867763    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"d7436000ac04ff26222ea5a272d6bf8de3d44865f7d1705a41aebffd964ae6ee"}
Oct 30 10:44:24.872334 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.872307    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rnv69" event={"ID":"1906564b-67cc-491d-b1f7-2c014046a18d","Type":"ContainerStarted","Data":"733d18f39bfd6e8244bf38c845bebaa833b6cd481466af9683f040a5095e939f"}
Oct 30 10:44:24.875838 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.875812    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q6ddt" event={"ID":"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657","Type":"ContainerStarted","Data":"0486f6c8dac9a33be956b33582bcd1aceb4e726edee6f89f4d452115f4e3126e"}
Oct 30 10:44:24.937654 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:24.937600    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" podStartSLOduration=1.9375811600000001 podStartE2EDuration="1.93758116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:24.935260274 +0000 UTC m=+3.662315420" watchObservedRunningTime="2026-10-30 10:44:24.93758116 +0000 UTC m=+3.664636304"
Oct 30 10:44:25.396763 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.395936    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:25.396763 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.395989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:25.396763 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.396176    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:25.396763 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.396238    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:44:27.396218275 +0000 UTC m=+6.123273409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:25.396763 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.396632    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:25.396763 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.396684    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret podName:bc612ba8-81c6-4d10-aecf-f8efdc6fea4f nodeName:}" failed. No retries permitted until 2026-10-30 10:44:27.396667614 +0000 UTC m=+6.123722750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret") pod "global-pull-secret-syncer-lkmz6" (UID: "bc612ba8-81c6-4d10-aecf-f8efdc6fea4f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:25.496730 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.496698    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:25.496905 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.496839    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 10:44:25.496905 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.496861    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 10:44:25.496905 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.496873    2584 projected.go:196] Error preparing data for projected volume kube-api-access-b42qs for pod openshift-network-diagnostics/network-check-target-ccnv5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:25.497076 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.496922    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs podName:167b7d19-70a6-43c4-a2c2-4386a574d3c3 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:27.496905891 +0000 UTC m=+6.223961010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b42qs" (UniqueName: "kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs") pod "network-check-target-ccnv5" (UID: "167b7d19-70a6-43c4-a2c2-4386a574d3c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:25.738351 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.738252    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 10:39:24 +0000 UTC" deadline="2028-04-23 06:00:43.386268158 +0000 UTC"
Oct 30 10:44:25.738351 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.738292    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12979h16m17.64798105s"
Oct 30 10:44:25.804622 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.803381    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:25.804622 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.803513    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:25.804622 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.803914    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:25.804622 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.804008    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:25.804622 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.804082    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:25.804622 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:25.804191    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:25.895497 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.895433    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"356d8c2a38ca6b7521952074b996a2cc","Type":"ContainerStarted","Data":"6d9f3be0b99abc8668e73a5030faf4aaf0443221aa9cab3695b65469f067df82"}
Oct 30 10:44:25.936242 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:25.936162    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" podStartSLOduration=2.936145575 podStartE2EDuration="2.936145575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:25.936015378 +0000 UTC m=+4.663070517" watchObservedRunningTime="2026-10-30 10:44:25.936145575 +0000 UTC m=+4.663200720"
Oct 30 10:44:27.416318 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.416274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:27.416894 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.416331    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:27.416894 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.416462    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:27.416894 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.416528    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:44:31.416508618 +0000 UTC m=+10.143563751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:27.417225 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.416946    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:27.417225 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.417007    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret podName:bc612ba8-81c6-4d10-aecf-f8efdc6fea4f nodeName:}" failed. No retries permitted until 2026-10-30 10:44:31.416989303 +0000 UTC m=+10.144044438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret") pod "global-pull-secret-syncer-lkmz6" (UID: "bc612ba8-81c6-4d10-aecf-f8efdc6fea4f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:27.517339 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.517304    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:27.517550 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.517527    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 10:44:27.517550 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.517549    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 10:44:27.517744 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.517563    2584 projected.go:196] Error preparing data for projected volume kube-api-access-b42qs for pod openshift-network-diagnostics/network-check-target-ccnv5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:27.517744 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.517638    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs podName:167b7d19-70a6-43c4-a2c2-4386a574d3c3 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:31.517620134 +0000 UTC m=+10.244675268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b42qs" (UniqueName: "kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs") pod "network-check-target-ccnv5" (UID: "167b7d19-70a6-43c4-a2c2-4386a574d3c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:27.800152 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.799187    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:27.800152 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.799257    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:27.800152 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.799265    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:27.800152 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.799368    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:27.800152 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.799479    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:27.800152 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:27.799563    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:27.914411 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.913562    2584 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="c74f5d83716017d6f53c6bc12343be55fc06c240df93484ea630662362cf6e55" exitCode=0
Oct 30 10:44:27.914411 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:27.913612    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerDied","Data":"c74f5d83716017d6f53c6bc12343be55fc06c240df93484ea630662362cf6e55"}
Oct 30 10:44:28.920077 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:28.920020    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q6ddt" event={"ID":"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657","Type":"ContainerStarted","Data":"23a1987d71c281a2514c589d874eca870b38946162d3efcad40c934a336e6ca8"}
Oct 30 10:44:28.962393 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:28.962348    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q6ddt" podStartSLOduration=2.125460363 podStartE2EDuration="5.96232693s" totalImagesPullingTime="3.836866567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:28.962246376 +0000 UTC m=+7.689301522" watchObservedRunningTime="2026-10-30 10:44:28.96232693 +0000 UTC m=+7.689382073"
Oct 30 10:44:29.800076 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:29.800049    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:29.800232 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:29.800203    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:29.800317 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:29.800304    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:29.800411 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:29.800389    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:29.800477 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:29.800464    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:29.800558 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:29.800540    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:29.928291 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:29.928256    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz" event={"ID":"984f643d-51d1-4142-b244-f8f0980a055f","Type":"ContainerStarted","Data":"43f13fde03671c2ede6c3ac2297fd86c90194b5880b9361e8f4fa36ca270f65c"}
Oct 30 10:44:29.932546 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:29.932518    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxz95" event={"ID":"7add3842-20f6-49eb-89ad-02078f7767ef","Type":"ContainerStarted","Data":"9cf16e5556ad23af4116b63f3336806ff1e96cbb15d2e97036b00902a891bbad"}
Oct 30 10:44:31.386999 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.386959    2584 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 10:44:31.459995 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.459920    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:31.460155 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.460034    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:31.460233 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.460184    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:31.460290 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.460248    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret podName:bc612ba8-81c6-4d10-aecf-f8efdc6fea4f nodeName:}" failed. No retries permitted until 2026-10-30 10:44:39.460228562 +0000 UTC m=+18.187283697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret") pod "global-pull-secret-syncer-lkmz6" (UID: "bc612ba8-81c6-4d10-aecf-f8efdc6fea4f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:31.460361 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.460320    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:31.460361 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.460357    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:44:39.460344694 +0000 UTC m=+18.187399828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:31.560956 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.560920    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:31.561159 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.561072    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 10:44:31.561159 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.561089    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 10:44:31.561159 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.561100    2584 projected.go:196] Error preparing data for projected volume kube-api-access-b42qs for pod openshift-network-diagnostics/network-check-target-ccnv5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:31.561330 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.561162    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs podName:167b7d19-70a6-43c4-a2c2-4386a574d3c3 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:39.561143628 +0000 UTC m=+18.288198764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b42qs" (UniqueName: "kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs") pod "network-check-target-ccnv5" (UID: "167b7d19-70a6-43c4-a2c2-4386a574d3c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:31.731875 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.731718    2584 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T10:44:31.386980466Z","UUID":"b934c4fb-ccb5-42ad-a1d1-fb10348edb7c","Handler":null,"Name":"","Endpoint":""}
Oct 30 10:44:31.738548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.737322    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 10:44:31.740076 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.738707    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 10:44:31.798491 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.798446    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:31.798640 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.798600    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:31.799043 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.799014    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:31.799160 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.799133    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:31.799240 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.799173    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:31.800558 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:31.800513    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:31.942155 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:31.942115    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz" event={"ID":"984f643d-51d1-4142-b244-f8f0980a055f","Type":"ContainerStarted","Data":"c265b0c620ba4a4bd7e88a342f316473bc0a37195038e4d1b1dd86591ec3a66d"}
Oct 30 10:44:32.079172 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.079077    2584 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:32.079686 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.079659    2584 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:32.121368 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.121311    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qxz95" podStartSLOduration=5.087842636 podStartE2EDuration="10.121294504s" totalImagesPullingTime="5.033451868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:29.973961571 +0000 UTC m=+8.701016716" watchObservedRunningTime="2026-10-30 10:44:32.121294504 +0000 UTC m=+10.848349647"
Oct 30 10:44:32.946756 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.946552    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-887wc" event={"ID":"ded181d7-f3c4-4324-946f-2770e8e3d0fd","Type":"ContainerStarted","Data":"d21dbb736fc963670f73dfb34054dcf9d7b13d7704cd47230fe6405981d4fab7"}
Oct 30 10:44:32.949752 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.949722    2584 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="7d82ffd81a0b526c5a74fd866902288286430c598e26bb3d32cd2ead09aa69d5" exitCode=0
Oct 30 10:44:32.949871 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.949824    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerDied","Data":"7d82ffd81a0b526c5a74fd866902288286430c598e26bb3d32cd2ead09aa69d5"}
Oct 30 10:44:32.953331 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.953307    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz" event={"ID":"984f643d-51d1-4142-b244-f8f0980a055f","Type":"ContainerStarted","Data":"4e2ba5870e802a57c1b5e357cb0a8aa5b7bc4277ff693b4ce1510485ebe1fcbd"}
Oct 30 10:44:32.955567 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.955541    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2c8hg" event={"ID":"1114f0eb-6317-4004-88fa-55f60fc991a5","Type":"ContainerStarted","Data":"68044acb83c9bc97d7d9f48fd4cbdb8364a9777580966ad467c27fee84169ffc"}
Oct 30 10:44:32.988454 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:32.988408    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-887wc" podStartSLOduration=2.6615816949999997 podStartE2EDuration="10.988395624s" totalImagesPullingTime="8.326813929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:32.987786516 +0000 UTC m=+11.714841878" watchObservedRunningTime="2026-10-30 10:44:32.988395624 +0000 UTC m=+11.715450766"
Oct 30 10:44:33.081544 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:33.081486    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-2c8hg" podStartSLOduration=2.913646157 podStartE2EDuration="11.081468176s" totalImagesPullingTime="8.167822019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:33.080706792 +0000 UTC m=+11.807761938" watchObservedRunningTime="2026-10-30 10:44:33.081468176 +0000 UTC m=+11.808523321"
Oct 30 10:44:33.125511 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:33.125459    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz" podStartSLOduration=1.949542326 podStartE2EDuration="10.125441786s" totalImagesPullingTime="8.17589946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:33.125049861 +0000 UTC m=+11.852105001" watchObservedRunningTime="2026-10-30 10:44:33.125441786 +0000 UTC m=+11.852496931"
Oct 30 10:44:33.797083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:33.797047    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:33.797247 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:33.797095    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:33.797247 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:33.797173    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:33.797374 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:33.797283    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:33.797700 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:33.797675    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:33.797816 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:33.797758    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:33.962082 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:33.961335    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2kxdq" event={"ID":"f1cb922d-534b-4fe0-8930-b115b820d656","Type":"ContainerStarted","Data":"1cbfbe62920ea38867a9e31fb340eda5b932581a88e8d88d118e09b5691c3daa"}
Oct 30 10:44:34.965742 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:34.965657    2584 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="23869dd34f996ae015bf48d600bab87cc87fc48bd8b9ae57156690356392a037" exitCode=0
Oct 30 10:44:34.966437 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:34.965743    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerDied","Data":"23869dd34f996ae015bf48d600bab87cc87fc48bd8b9ae57156690356392a037"}
Oct 30 10:44:34.967981 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:34.967888    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rnv69" event={"ID":"1906564b-67cc-491d-b1f7-2c014046a18d","Type":"ContainerStarted","Data":"335a37f727f09b431b82413a36cad8abe661ea6cf1d130eff2dad1bf9587070e"}
Oct 30 10:44:35.041564 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:35.041518    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2kxdq" podStartSLOduration=4.852928444 podStartE2EDuration="13.041499606s" totalImagesPullingTime="8.188571162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:34.001985294 +0000 UTC m=+12.729040438" watchObservedRunningTime="2026-10-30 10:44:35.041499606 +0000 UTC m=+13.768554752"
Oct 30 10:44:35.084383 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:35.084346    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rnv69" podStartSLOduration=1.6076523539999998 podStartE2EDuration="12.084334738s" totalImagesPullingTime="10.476682384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:35.084034051 +0000 UTC m=+13.811089183" watchObservedRunningTime="2026-10-30 10:44:35.084334738 +0000 UTC m=+13.811389921"
Oct 30 10:44:35.797341 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:35.797317    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:35.797448 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:35.797326    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:35.797448 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:35.797436    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:35.797556 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:35.797498    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:35.797556 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:35.797548    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:35.797843 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:35.797816    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:35.972763 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:35.972733    2584 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="39310ff80c1f589aa10599656e8fe7f452cfbc2039c82572ea0928ae8f715340" exitCode=0
Oct 30 10:44:35.973362 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:35.972816    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerDied","Data":"39310ff80c1f589aa10599656e8fe7f452cfbc2039c82572ea0928ae8f715340"}
Oct 30 10:44:37.796995 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:37.796772    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:37.797364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:37.796777    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:37.797364 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:37.796825    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:37.797364 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:37.797130    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:37.797364 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:37.797265    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:37.797364 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:37.797346    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:37.980520 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:37.980454    2584 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="3c7bbed30f3252bf5a62a48c879e26b104cd0ac270610af06a05628d9dbe0ddf" exitCode=0
Oct 30 10:44:37.980520 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:37.980509    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerDied","Data":"3c7bbed30f3252bf5a62a48c879e26b104cd0ac270610af06a05628d9dbe0ddf"}
Oct 30 10:44:38.985546 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:38.985510    2584 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="355a1810a1f865b52692daf925904f3e88341445dca434dea6ef25755b2e648e" exitCode=0
Oct 30 10:44:38.985992 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:38.985559    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerDied","Data":"355a1810a1f865b52692daf925904f3e88341445dca434dea6ef25755b2e648e"}
Oct 30 10:44:39.530272 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.530244    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:39.530365 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.530323    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:39.530461 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.530378    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:39.530461 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.530411    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:39.530461 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.530435    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:44:55.530417973 +0000 UTC m=+34.257473101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 10:44:39.530461 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.530450    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret podName:bc612ba8-81c6-4d10-aecf-f8efdc6fea4f nodeName:}" failed. No retries permitted until 2026-10-30 10:44:55.530443886 +0000 UTC m=+34.257499004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret") pod "global-pull-secret-syncer-lkmz6" (UID: "bc612ba8-81c6-4d10-aecf-f8efdc6fea4f") : object "kube-system"/"original-pull-secret" not registered
Oct 30 10:44:39.630830 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.630699    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:39.630924 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.630843    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 10:44:39.630924 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.630861    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 10:44:39.630924 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.630872    2584 projected.go:196] Error preparing data for projected volume kube-api-access-b42qs for pod openshift-network-diagnostics/network-check-target-ccnv5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:39.630924 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.630914    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs podName:167b7d19-70a6-43c4-a2c2-4386a574d3c3 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:55.630899669 +0000 UTC m=+34.357954805 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b42qs" (UniqueName: "kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs") pod "network-check-target-ccnv5" (UID: "167b7d19-70a6-43c4-a2c2-4386a574d3c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 10:44:39.797200 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.797174    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:39.797298 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.797204    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:39.797338 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.797288    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:39.797411 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.797391    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:39.797471 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.797460    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:39.797584 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:39.797567    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:39.991217 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.991152    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4k458" event={"ID":"dc78343b-907d-4222-b27d-7705a2a0f19e","Type":"ContainerStarted","Data":"c1bdcbbaefb7f1782cb3b388a386203cea083568b30f1a556dd78157d188b2b0"}
Oct 30 10:44:39.994467 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.994443    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"aed1cb6e2b3cb723ca6ff52bade320cb4ebb30fbc0ade24a64c9dcb0cabeefa8"}
Oct 30 10:44:39.994575 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.994499    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"884ccc45e80cda8910e5816313c20b154b48b77e65c0d5ea4049b4029a1a2d4e"}
Oct 30 10:44:39.994575 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.994529    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"39a2114c5236c4cbe36601b3d1d678d0d677f6b43b19f3bf41666e3cf932556c"}
Oct 30 10:44:39.994575 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.994556    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"1f6aa51c75636d951ad4bafab8f733468e04e7b167eda8f8ba0bb36139e0195c"}
Oct 30 10:44:39.994682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.994586    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"5071846410d54dc25b9578e3279e6c1ee17322caa37318b81c89903b5bd4ed11"}
Oct 30 10:44:39.994682 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:39.994614    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"222e76a04a2f1209e1d5c366334abb2d490f18de5758f0e9d706e67efce2a17e"}
Oct 30 10:44:40.037490 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:40.037449    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4k458" podStartSLOduration=6.277952596 podStartE2EDuration="17.037437424s" totalImagesPullingTime="10.759484828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:40.037142536 +0000 UTC m=+18.764197676" watchObservedRunningTime="2026-10-30 10:44:40.037437424 +0000 UTC m=+18.764492563"
Oct 30 10:44:41.797535 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:41.797515    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:41.797844 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:41.797674    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:41.797844 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:41.797686    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:41.797926 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:41.797852    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:41.797926 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:41.797874    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:41.798010 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:41.797961    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:42.001555 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:42.001527    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"580eda7572a2e188b87bedd9ec8a78bc9c6bfc5aeda852e55462b8fa07b29019"}
Oct 30 10:44:43.796984 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:43.796949    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:43.796984 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:43.796967    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:43.797861 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:43.796966    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:43.797861 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:43.797083    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:43.797861 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:43.797208    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:43.797861 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:43.797297    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:44.009974 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:44.009938    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" event={"ID":"eb883d09-b098-410d-bc1d-f68e24021d17","Type":"ContainerStarted","Data":"1d2f624d7abd269bd1e190302e92f6d0fa96537f067ecc89343a1acd48451d4d"}
Oct 30 10:44:44.010228 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:44.010208    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:44.024530 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:44.024506    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:44.096569 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:44.096487    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r" podStartSLOduration=5.688230778 podStartE2EDuration="21.096474038s" totalImagesPullingTime="15.40824326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:44.095845538 +0000 UTC m=+22.822900679" watchObservedRunningTime="2026-10-30 10:44:44.096474038 +0000 UTC m=+22.823529178"
Oct 30 10:44:45.012856 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.012827    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:45.013548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.012921    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:45.028452 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.028425    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:44:45.186269 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.186237    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:45.186438 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.186384    2584 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 10:44:45.186774 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.186746    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:44:45.797382 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.797345    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:45.797382 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.797370    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:45.797664 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:45.797445    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:45.797664 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.797497    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:45.797664 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:45.797563    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:45.797783 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:45.797654    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:45.936005 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.935979    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lkmz6"]
Oct 30 10:44:45.939242 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.939218    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ccnv5"]
Oct 30 10:44:45.939714 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:45.939692    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hr78s"]
Oct 30 10:44:46.014163 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:46.014138    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:46.014163 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:46.014139    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:46.014521 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:46.014233    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:46.014521 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:46.014312    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:46.014521 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:46.014371    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:46.014758 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:46.014740    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:47.797442 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:47.797241    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:47.797851 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:47.797257    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:47.797851 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:47.797529    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:47.797851 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:47.797327    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:47.797851 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:47.797599    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:47.797851 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:47.797717    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:49.797133 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:49.797098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:49.797133 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:49.797115    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:49.797577 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:49.797226    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:49.797577 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:49.797257    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:49.797577 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:49.797297    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:49.797577 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:49.797396    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:51.797771 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:51.797744    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:51.798089 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:51.797826    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccnv5" podUID="167b7d19-70a6-43c4-a2c2-4386a574d3c3"
Oct 30 10:44:51.798089 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:51.797914    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:51.798089 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:51.798012    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hr78s" podUID="0df37afa-77d5-4bf7-bc83-0d7636453bda"
Oct 30 10:44:51.798089 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:51.798079    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:51.798225 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:51.798153    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lkmz6" podUID="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f"
Oct 30 10:44:52.116928 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.116909    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeReady"
Oct 30 10:44:52.117046 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.116993    2584 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 10:44:52.165619 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.165596    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8pgqv"]
Oct 30 10:44:52.216222 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.216146    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xfp2x"]
Oct 30 10:44:52.216411 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.216258    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8pgqv"
Oct 30 10:44:52.231287 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.231260    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.231411 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.231360    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.231688 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.231666    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-r9ns7"]
Oct 30 10:44:52.231799 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.231691    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xfp2x"
Oct 30 10:44:52.243296 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.243255    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-794b85596b-gvxrs"]
Oct 30 10:44:52.244052 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.243504    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-r9ns7"
Oct 30 10:44:52.256395 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.256374    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"]
Oct 30 10:44:52.256506 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.256489    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.271046 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.271015    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 10:44:52.271385 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.271369    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 10:44:52.271385 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.271372    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 10:44:52.271475 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.271435    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gnsmw\""
Oct 30 10:44:52.274398 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.274382    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-v69fn"]
Oct 30 10:44:52.274557 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.274543    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.289561 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.289545    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-65668b76cf-ckhgx"]
Oct 30 10:44:52.289658 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.289644    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.303599 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.303583    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"]
Oct 30 10:44:52.303712 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.303698    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.317742 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.317727    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"]
Oct 30 10:44:52.317849 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.317836    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.328954 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.328937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hct8\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329065 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.328961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8zlc\" (UniqueName: \"kubernetes.io/projected/e3cc17ea-0cec-4a71-882d-2df0457dd957-kube-api-access-h8zlc\") pod \"kas-connection-checker-685895c7c9-r9ns7\" (UID: \"e3cc17ea-0cec-4a71-882d-2df0457dd957\") " pod="kube-system/kas-connection-checker-685895c7c9-r9ns7"
Oct 30 10:44:52.329065 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.328981    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329065 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329184 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329184 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329181    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329260 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329198    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329260 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329214    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r272g\" (UniqueName: \"kubernetes.io/projected/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2-kube-api-access-r272g\") pod \"kas-connection-checker-685895c7c9-8pgqv\" (UID: \"f384be3a-0f13-4b0a-ba1b-d63d2095b9d2\") " pod="kube-system/kas-connection-checker-685895c7c9-8pgqv"
Oct 30 10:44:52.329343 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329325    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329389 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329343    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.329389 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.329359    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq49n\" (UniqueName: \"kubernetes.io/projected/418cb483-75bb-49b0-9c67-f3b70c26c2b4-kube-api-access-hq49n\") pod \"kas-connection-checker-685895c7c9-xfp2x\" (UID: \"418cb483-75bb-49b0-9c67-f3b70c26c2b4\") " pod="kube-system/kas-connection-checker-685895c7c9-xfp2x"
Oct 30 10:44:52.331529 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331513    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 10:44:52.331615 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331580    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.331615 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331607    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 10:44:52.331701 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331619    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.331701 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331622    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.331701 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331631    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.331788 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331710    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.331788 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331775    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.331937 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.331923    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"]
Oct 30 10:44:52.332054 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.332037    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.344003 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.343988    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 10:44:52.344106 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.344059    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.354975 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.354954    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"]
Oct 30 10:44:52.355087 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.354978    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.355087 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.355049    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 10:44:52.355190 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.355108    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.356100 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.356077    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 10:44:52.356245 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.356227    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.356314 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.356243    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 10:44:52.356314 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.356250    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.356459 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.356440    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 10:44:52.357216 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.357201    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 10:44:52.368706 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.368689    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"]
Oct 30 10:44:52.368810 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.368796    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"
Oct 30 10:44:52.372104 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372085    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 10:44:52.372186 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372120    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 10:44:52.372186 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372148    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 10:44:52.372290 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372204    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 10:44:52.372290 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372240    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 10:44:52.372290 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372255    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 10:44:52.372290 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372241    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 10:44:52.372290 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.372263    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 10:44:52.386974 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.386959    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"]
Oct 30 10:44:52.387095 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.387079    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:52.407053 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.407033    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67bdbf894c-qszt7"]
Oct 30 10:44:52.407141 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.407127    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.430065 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430043    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqwn6\" (UniqueName: \"kubernetes.io/projected/9e199587-19b7-4a4b-bdd0-761b7a6e475f-kube-api-access-cqwn6\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.430159 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430076    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.430159 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430093    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-service-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.430159 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430109    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.430159 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430153    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-tmp\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.430295 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430225    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvnqf\" (UniqueName: \"kubernetes.io/projected/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-kube-api-access-tvnqf\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.430295 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430252    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-trusted-ca\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.430295 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.430428 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430332    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.430428 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430387    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.430524 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430485    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e199587-19b7-4a4b-bdd0-761b7a6e475f-serving-cert\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.430524 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430520    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.430622 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430555    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlkjh\" (UniqueName: \"kubernetes.io/projected/1cbcd9b8-ce20-494c-9a66-475272f2a561-kube-api-access-hlkjh\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.430622 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430573    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.430622 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq49n\" (UniqueName: \"kubernetes.io/projected/418cb483-75bb-49b0-9c67-f3b70c26c2b4-kube-api-access-hq49n\") pod \"kas-connection-checker-685895c7c9-xfp2x\" (UID: \"418cb483-75bb-49b0-9c67-f3b70c26c2b4\") " pod="kube-system/kas-connection-checker-685895c7c9-xfp2x"
Oct 30 10:44:52.430622 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430612    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.430780 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430651    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7333057e-e1db-4970-bf86-eede9e94c06a-serving-cert\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.430870 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430839    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hct8\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.430986 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430882    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zlc\" (UniqueName: \"kubernetes.io/projected/e3cc17ea-0cec-4a71-882d-2df0457dd957-kube-api-access-h8zlc\") pod \"kas-connection-checker-685895c7c9-r9ns7\" (UID: \"e3cc17ea-0cec-4a71-882d-2df0457dd957\") " pod="kube-system/kas-connection-checker-685895c7c9-r9ns7"
Oct 30 10:44:52.430986 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430908    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-config\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.430986 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v75s\" (UniqueName: \"kubernetes.io/projected/04a41ddb-fcb6-49bf-8468-deccd08a460c-kube-api-access-8v75s\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.430986 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430963    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7333057e-e1db-4970-bf86-eede9e94c06a-config\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.430989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431012    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54d74\" (UniqueName: \"kubernetes.io/projected/9bd59176-1f2c-4169-a6bc-45ca82e270eb-kube-api-access-54d74\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431049    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-serving-cert\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431074    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a41ddb-fcb6-49bf-8468-deccd08a460c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-snapshots\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431150    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431236    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a41ddb-fcb6-49bf-8468-deccd08a460c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431266    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1cbcd9b8-ce20-494c-9a66-475272f2a561-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r272g\" (UniqueName: \"kubernetes.io/projected/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2-kube-api-access-r272g\") pod \"kas-connection-checker-685895c7c9-8pgqv\" (UID: \"f384be3a-0f13-4b0a-ba1b-d63d2095b9d2\") " pod="kube-system/kas-connection-checker-685895c7c9-8pgqv"
Oct 30 10:44:52.431304 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431293    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431325    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431333    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-default-certificate\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431386    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ncc7\" (UniqueName: \"kubernetes.io/projected/7333057e-e1db-4970-bf86-eede9e94c06a-kube-api-access-6ncc7\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431419    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.431522    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-stats-auth\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.431591    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.431608    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:44:52.431815 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.431654    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:52.931637247 +0000 UTC m=+31.658692367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:44:52.432215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432015    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"]
Oct 30 10:44:52.432215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432127    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.432332 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432315    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.432386 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432319    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.432386 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432345    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.432481 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432385    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 10:44:52.432481 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432389    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.432481 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432400    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 10:44:52.432481 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432452    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.432661 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432484    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.432661 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.432524    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 10:44:52.434075 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.434057    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.434148 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.434079    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.452956 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.452934    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq49n\" (UniqueName: \"kubernetes.io/projected/418cb483-75bb-49b0-9c67-f3b70c26c2b4-kube-api-access-hq49n\") pod \"kas-connection-checker-685895c7c9-xfp2x\" (UID: \"418cb483-75bb-49b0-9c67-f3b70c26c2b4\") " pod="kube-system/kas-connection-checker-685895c7c9-xfp2x"
Oct 30 10:44:52.453215 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.453196    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.453765 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.453749    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8pgqv"]
Oct 30 10:44:52.453844 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.453781    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mx9gc"]
Oct 30 10:44:52.453895 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.453865    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"
Oct 30 10:44:52.454467 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.454449    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hct8\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.454467 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.454460    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r272g\" (UniqueName: \"kubernetes.io/projected/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2-kube-api-access-r272g\") pod \"kas-connection-checker-685895c7c9-8pgqv\" (UID: \"f384be3a-0f13-4b0a-ba1b-d63d2095b9d2\") " pod="kube-system/kas-connection-checker-685895c7c9-8pgqv"
Oct 30 10:44:52.454567 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.454504    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zlc\" (UniqueName: \"kubernetes.io/projected/e3cc17ea-0cec-4a71-882d-2df0457dd957-kube-api-access-h8zlc\") pod \"kas-connection-checker-685895c7c9-r9ns7\" (UID: \"e3cc17ea-0cec-4a71-882d-2df0457dd957\") " pod="kube-system/kas-connection-checker-685895c7c9-r9ns7"
Oct 30 10:44:52.472105 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.472091    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xfp2x"]
Oct 30 10:44:52.472176 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.472116    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7kq9h"]
Oct 30 10:44:52.472212 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.472198    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.472511 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.472493    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 10:44:52.472631 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.472617    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 10:44:52.472693 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.472678    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-8g6wd\""
Oct 30 10:44:52.483583 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483563    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-r9ns7"]
Oct 30 10:44:52.483654 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483596    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"]
Oct 30 10:44:52.483654 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483643    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65668b76cf-ckhgx"]
Oct 30 10:44:52.483712 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483663    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"]
Oct 30 10:44:52.483712 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483676    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"]
Oct 30 10:44:52.483712 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483688    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"]
Oct 30 10:44:52.483712 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483705    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"]
Oct 30 10:44:52.483843 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483730    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"]
Oct 30 10:44:52.483843 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483761    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.483843 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483777    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mx9gc"]
Oct 30 10:44:52.483843 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483803    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-v69fn"]
Oct 30 10:44:52.483843 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483821    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-794b85596b-gvxrs"]
Oct 30 10:44:52.483992 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483833    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"]
Oct 30 10:44:52.483992 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483870    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"]
Oct 30 10:44:52.483992 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483889    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7kq9h"]
Oct 30 10:44:52.483992 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.483908    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67bdbf894c-qszt7"]
Oct 30 10:44:52.532088 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532069    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqwn6\" (UniqueName: \"kubernetes.io/projected/9e199587-19b7-4a4b-bdd0-761b7a6e475f-kube-api-access-cqwn6\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.532188 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532108    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jnkk\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.532188 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532134    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.532291 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-service-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.532291 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532221    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn5vl\" (UniqueName: \"kubernetes.io/projected/a179c33d-aecd-4b72-b907-37f82a9aec15-kube-api-access-qn5vl\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.532291 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.532444 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532348    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-tmp\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.532444 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532379    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.532444 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532406    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.532444 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.532442    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.032425399 +0000 UTC m=+31.759480522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : configmap references non-existent config key: service-ca.crt
Oct 30 10:44:52.532662 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532515    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvnqf\" (UniqueName: \"kubernetes.io/projected/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-kube-api-access-tvnqf\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.532662 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532550    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.532662 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532589    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-trusted-ca\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.532662 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532621    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e199587-19b7-4a4b-bdd0-761b7a6e475f-serving-cert\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532728    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-tmp\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532744    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532746    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-service-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532801    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlkjh\" (UniqueName: \"kubernetes.io/projected/1cbcd9b8-ce20-494c-9a66-475272f2a561-kube-api-access-hlkjh\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532819    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532827    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532837    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.532852 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.532843    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532891    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.532898    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.032881853 +0000 UTC m=+31.759936977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532911    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532960    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.532989    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533063    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533095    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7333057e-e1db-4970-bf86-eede9e94c06a-serving-cert\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533122    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8grgp\" (UniqueName: \"kubernetes.io/projected/fa764bd7-2feb-4a11-880e-70118d1ac639-kube-api-access-8grgp\") pod \"network-check-source-6f78d59f99-w7wqz\" (UID: \"fa764bd7-2feb-4a11-880e-70118d1ac639\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-config\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.533175 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.533160    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533182    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v75s\" (UniqueName: \"kubernetes.io/projected/04a41ddb-fcb6-49bf-8468-deccd08a460c-kube-api-access-8v75s\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.533211    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls podName:1cbcd9b8-ce20-494c-9a66-475272f2a561 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.033194351 +0000 UTC m=+31.760249476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lw745" (UID: "1cbcd9b8-ce20-494c-9a66-475272f2a561") : secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533229    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7333057e-e1db-4970-bf86-eede9e94c06a-config\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533269    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54d74\" (UniqueName: \"kubernetes.io/projected/9bd59176-1f2c-4169-a6bc-45ca82e270eb-kube-api-access-54d74\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533347    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-serving-cert\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533386    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533418    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a41ddb-fcb6-49bf-8468-deccd08a460c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533486    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-snapshots\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.533620 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533530    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.533932 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533742    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.533932 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533768    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2hnd\" (UniqueName: \"kubernetes.io/projected/b8b2a817-e3d9-4361-98cd-c0a40c93b84d-kube-api-access-z2hnd\") pod \"volume-data-source-validator-58c9f5f46d-5xkvn\" (UID: \"b8b2a817-e3d9-4361-98cd-c0a40c93b84d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"
Oct 30 10:44:52.533932 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533795    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.533932 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533850    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a41ddb-fcb6-49bf-8468-deccd08a460c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.533932 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533879    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1cbcd9b8-ce20-494c-9a66-475272f2a561-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.533932 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.533888    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-trusted-ca\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.534248 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534101    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-default-certificate\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.534248 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534148    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ncc7\" (UniqueName: \"kubernetes.io/projected/7333057e-e1db-4970-bf86-eede9e94c06a-kube-api-access-6ncc7\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.534248 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534185    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7333057e-e1db-4970-bf86-eede9e94c06a-config\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.534386 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534348    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-config\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.534661 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534642    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1cbcd9b8-ce20-494c-9a66-475272f2a561-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.534743 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534691    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.534806 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534752    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-stats-auth\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.534806 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534795    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3138c05b-7ff7-463e-9d91-ea7fab54a541-nginx-conf\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:52.534918 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534829    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.534967 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534951    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-snapshots\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.535017 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.534989    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a41ddb-fcb6-49bf-8468-deccd08a460c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.535406 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.535382    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7333057e-e1db-4970-bf86-eede9e94c06a-serving-cert\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.535480 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.535448    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e199587-19b7-4a4b-bdd0-761b7a6e475f-serving-cert\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.536813 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.536786    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a41ddb-fcb6-49bf-8468-deccd08a460c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.536897 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.536812    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-serving-cert\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.537300 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.537279    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-default-certificate\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.537300 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.537289    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-stats-auth\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.543385 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.543368    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8pgqv"
Oct 30 10:44:52.556550 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.556535    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xfp2x"
Oct 30 10:44:52.561887 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.561868    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqwn6\" (UniqueName: \"kubernetes.io/projected/9e199587-19b7-4a4b-bdd0-761b7a6e475f-kube-api-access-cqwn6\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.562432 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.562411    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlkjh\" (UniqueName: \"kubernetes.io/projected/1cbcd9b8-ce20-494c-9a66-475272f2a561-kube-api-access-hlkjh\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:52.562561 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.562541    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvnqf\" (UniqueName: \"kubernetes.io/projected/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-kube-api-access-tvnqf\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.563387 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.563362    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ncc7\" (UniqueName: \"kubernetes.io/projected/7333057e-e1db-4970-bf86-eede9e94c06a-kube-api-access-6ncc7\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.563482 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.563448    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54d74\" (UniqueName: \"kubernetes.io/projected/9bd59176-1f2c-4169-a6bc-45ca82e270eb-kube-api-access-54d74\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:52.563612 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.563594    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v75s\" (UniqueName: \"kubernetes.io/projected/04a41ddb-fcb6-49bf-8468-deccd08a460c-kube-api-access-8v75s\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.572489 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.572472    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 10:44:52.572722 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.572704    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 10:44:52.572792 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.572778    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-r88l6\""
Oct 30 10:44:52.572834 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.572786    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgw7c\""
Oct 30 10:44:52.575187 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.575170    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-r9ns7"
Oct 30 10:44:52.610120 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.610098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.637040    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grgp\" (UniqueName: \"kubernetes.io/projected/fa764bd7-2feb-4a11-880e-70118d1ac639-kube-api-access-8grgp\") pod \"network-check-source-6f78d59f99-w7wqz\" (UID: \"fa764bd7-2feb-4a11-880e-70118d1ac639\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.637098    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.637131    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.637162    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq8tw\" (UniqueName: \"kubernetes.io/projected/d88e5148-03db-43f8-a46f-107d8d42af0f-kube-api-access-vq8tw\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.637195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.637222    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2hnd\" (UniqueName: \"kubernetes.io/projected/b8b2a817-e3d9-4361-98cd-c0a40c93b84d-kube-api-access-z2hnd\") pod \"volume-data-source-validator-58c9f5f46d-5xkvn\" (UID: \"b8b2a817-e3d9-4361-98cd-c0a40c93b84d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638076    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638112    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d88e5148-03db-43f8-a46f-107d8d42af0f-cert\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638175    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3138c05b-7ff7-463e-9d91-ea7fab54a541-nginx-conf\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638205    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638242    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jnkk\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638277    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn5vl\" (UniqueName: \"kubernetes.io/projected/a179c33d-aecd-4b72-b907-37f82a9aec15-kube-api-access-qn5vl\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638307    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638346    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638373    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.640501 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fba2151-22de-4a03-afcb-1d89b9c32a2b-config-volume\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638440    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qshr8\" (UniqueName: \"kubernetes.io/projected/2fba2151-22de-4a03-afcb-1d89b9c32a2b-kube-api-access-qshr8\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638466    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638537    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba2151-22de-4a03-afcb-1d89b9c32a2b-tmp-dir\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638575    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638606    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.638739    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.638795    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.138778125 +0000 UTC m=+31.865833256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.638904    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.639598    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3138c05b-7ff7-463e-9d91-ea7fab54a541-nginx-conf\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.639680    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.639725    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.639742    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.639801    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.139783324 +0000 UTC m=+31.866838443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.640396    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.641083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.640452    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.140440395 +0000 UTC m=+31.867495513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:44:52.641893 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.641605    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.642636 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.642582    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.650039 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.645928    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.650851 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.650539    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.650851 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.650653    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:44:52.665726 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.665667    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:44:52.675531 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.675504    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.675807 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.675758    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2hnd\" (UniqueName: \"kubernetes.io/projected/b8b2a817-e3d9-4361-98cd-c0a40c93b84d-kube-api-access-z2hnd\") pod \"volume-data-source-validator-58c9f5f46d-5xkvn\" (UID: \"b8b2a817-e3d9-4361-98cd-c0a40c93b84d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"
Oct 30 10:44:52.677547 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.676591    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn5vl\" (UniqueName: \"kubernetes.io/projected/a179c33d-aecd-4b72-b907-37f82a9aec15-kube-api-access-qn5vl\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:52.683477 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.678144    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jnkk\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:52.683477 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.681150    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8grgp\" (UniqueName: \"kubernetes.io/projected/fa764bd7-2feb-4a11-880e-70118d1ac639-kube-api-access-8grgp\") pod \"network-check-source-6f78d59f99-w7wqz\" (UID: \"fa764bd7-2feb-4a11-880e-70118d1ac639\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"
Oct 30 10:44:52.687641 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.687517    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:44:52.706340 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.704636    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"
Oct 30 10:44:52.740980 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.740096    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba2151-22de-4a03-afcb-1d89b9c32a2b-tmp-dir\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.740980 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.740239    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq8tw\" (UniqueName: \"kubernetes.io/projected/d88e5148-03db-43f8-a46f-107d8d42af0f-kube-api-access-vq8tw\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.740980 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.740348    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d88e5148-03db-43f8-a46f-107d8d42af0f-cert\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.743259 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.741505    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.743259 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.741574    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fba2151-22de-4a03-afcb-1d89b9c32a2b-config-volume\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.743259 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.741597    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qshr8\" (UniqueName: \"kubernetes.io/projected/2fba2151-22de-4a03-afcb-1d89b9c32a2b-kube-api-access-qshr8\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.743259 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.741982    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:44:52.743259 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.742063    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.242045339 +0000 UTC m=+31.969100457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:44:52.743259 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.742820    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fba2151-22de-4a03-afcb-1d89b9c32a2b-config-volume\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.743259 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.743056    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba2151-22de-4a03-afcb-1d89b9c32a2b-tmp-dir\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.759118 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.746834    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d88e5148-03db-43f8-a46f-107d8d42af0f-cert\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.784594 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.784525    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qshr8\" (UniqueName: \"kubernetes.io/projected/2fba2151-22de-4a03-afcb-1d89b9c32a2b-kube-api-access-qshr8\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:52.798623 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.787396    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq8tw\" (UniqueName: \"kubernetes.io/projected/d88e5148-03db-43f8-a46f-107d8d42af0f-kube-api-access-vq8tw\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.798623 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.798168    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"
Oct 30 10:44:52.816423 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.816396    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:44:52.888548 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.888522    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xfp2x"]
Oct 30 10:44:52.889336 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.889312    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8pgqv"]
Oct 30 10:44:52.891700 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:52.891638    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf384be3a_0f13_4b0a_ba1b_d63d2095b9d2.slice/crio-78c49b901c1a17ecb9be8a9af3c2b860357d0cd0e2b27b036f9b41b03a23cf28 WatchSource:0}: Error finding container 78c49b901c1a17ecb9be8a9af3c2b860357d0cd0e2b27b036f9b41b03a23cf28: Status 404 returned error can't find the container with id 78c49b901c1a17ecb9be8a9af3c2b860357d0cd0e2b27b036f9b41b03a23cf28
Oct 30 10:44:52.933688 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.933643    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-r9ns7"]
Oct 30 10:44:52.943188 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.943135    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:52.943406 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.943328    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:52.943406 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.943343    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:44:52.943406 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:52.943388    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:53.943371339 +0000 UTC m=+32.670426474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:44:52.958401 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.958378    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-v69fn"]
Oct 30 10:44:52.962074 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:52.962043    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e199587_19b7_4a4b_bdd0_761b7a6e475f.slice/crio-28555fb462459f87e6b3be52779d112f84230173a0f43e6b692179bc23a35d6d WatchSource:0}: Error finding container 28555fb462459f87e6b3be52779d112f84230173a0f43e6b692179bc23a35d6d: Status 404 returned error can't find the container with id 28555fb462459f87e6b3be52779d112f84230173a0f43e6b692179bc23a35d6d
Oct 30 10:44:52.992427 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:52.991371    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"]
Oct 30 10:44:52.996328 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:52.996302    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7333057e_e1db_4970_bf86_eede9e94c06a.slice/crio-af02cc1967f17431e480c5a28b0f6de45431f63d9e431bba6c130b0956e3d0f9 WatchSource:0}: Error finding container af02cc1967f17431e480c5a28b0f6de45431f63d9e431bba6c130b0956e3d0f9: Status 404 returned error can't find the container with id af02cc1967f17431e480c5a28b0f6de45431f63d9e431bba6c130b0956e3d0f9
Oct 30 10:44:53.008946 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.008921    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"]
Oct 30 10:44:53.012971 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:53.012938    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909137f2_6cf7_4fc8_8bcd_b0620b87aa5a.slice/crio-997deae2ef22fb6b0f9c791a7454c5aa9821388b03fd76c40b717f3f95c4df0c WatchSource:0}: Error finding container 997deae2ef22fb6b0f9c791a7454c5aa9821388b03fd76c40b717f3f95c4df0c: Status 404 returned error can't find the container with id 997deae2ef22fb6b0f9c791a7454c5aa9821388b03fd76c40b717f3f95c4df0c
Oct 30 10:44:53.029753 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.029725    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-r9ns7" event={"ID":"e3cc17ea-0cec-4a71-882d-2df0457dd957","Type":"ContainerStarted","Data":"91f94347ed9c3b23754ab3ea271fe0549d3f60a3ab0df7f4d1427868b99627db"}
Oct 30 10:44:53.031930 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.031911    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xfp2x" event={"ID":"418cb483-75bb-49b0-9c67-f3b70c26c2b4","Type":"ContainerStarted","Data":"ff0e1831cc43c694d6d09a2b2fbe9a14163550eee2d9fa8c5875c489c6b44861"}
Oct 30 10:44:53.032993 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.032941    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst" event={"ID":"7333057e-e1db-4970-bf86-eede9e94c06a","Type":"ContainerStarted","Data":"af02cc1967f17431e480c5a28b0f6de45431f63d9e431bba6c130b0956e3d0f9"}
Oct 30 10:44:53.034129 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.034106    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-v69fn" event={"ID":"9e199587-19b7-4a4b-bdd0-761b7a6e475f","Type":"ContainerStarted","Data":"28555fb462459f87e6b3be52779d112f84230173a0f43e6b692179bc23a35d6d"}
Oct 30 10:44:53.035594 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.035552    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" event={"ID":"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a","Type":"ContainerStarted","Data":"997deae2ef22fb6b0f9c791a7454c5aa9821388b03fd76c40b717f3f95c4df0c"}
Oct 30 10:44:53.037661 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.037613    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8pgqv" event={"ID":"f384be3a-0f13-4b0a-ba1b-d63d2095b9d2","Type":"ContainerStarted","Data":"78c49b901c1a17ecb9be8a9af3c2b860357d0cd0e2b27b036f9b41b03a23cf28"}
Oct 30 10:44:53.044052 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.044012    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"]
Oct 30 10:44:53.044330 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.044312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:53.044368 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.044359    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:53.044434 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.044425    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:53.044547 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.044537    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:44:54.044524302 +0000 UTC m=+32.771579432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : configmap references non-existent config key: service-ca.crt
Oct 30 10:44:53.044815 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.044802    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:44:53.044851 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.044839    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:44:54.044829093 +0000 UTC m=+32.771884224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:44:53.044883 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.044876    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:53.044913 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.044897    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls podName:1cbcd9b8-ce20-494c-9a66-475272f2a561 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:54.044890352 +0000 UTC m=+32.771945483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lw745" (UID: "1cbcd9b8-ce20-494c-9a66-475272f2a561") : secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:53.047463 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:53.047444    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b2a817_e3d9_4361_98cd_c0a40c93b84d.slice/crio-eb0076369a88562ee31eb074acf596cb34012dff875fdf385b0709be67695468 WatchSource:0}: Error finding container eb0076369a88562ee31eb074acf596cb34012dff875fdf385b0709be67695468: Status 404 returned error can't find the container with id eb0076369a88562ee31eb074acf596cb34012dff875fdf385b0709be67695468
Oct 30 10:44:53.048167 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.048149    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"]
Oct 30 10:44:53.052050 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:53.050622    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a41ddb_fcb6_49bf_8468_deccd08a460c.slice/crio-a166dcbe1caca639bf7ee98a7a9d253f88e1d58d3204bd32d6e5b0aea39db67c WatchSource:0}: Error finding container a166dcbe1caca639bf7ee98a7a9d253f88e1d58d3204bd32d6e5b0aea39db67c: Status 404 returned error can't find the container with id a166dcbe1caca639bf7ee98a7a9d253f88e1d58d3204bd32d6e5b0aea39db67c
Oct 30 10:44:53.125410 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.125387    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"]
Oct 30 10:44:53.128149 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:53.128128    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa764bd7_2feb_4a11_880e_70118d1ac639.slice/crio-2a552d464a9d49f9ac1dd857d287882eb8c4f367f58da3022c858f796ab898f9 WatchSource:0}: Error finding container 2a552d464a9d49f9ac1dd857d287882eb8c4f367f58da3022c858f796ab898f9: Status 404 returned error can't find the container with id 2a552d464a9d49f9ac1dd857d287882eb8c4f367f58da3022c858f796ab898f9
Oct 30 10:44:53.144931 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.144908    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:53.145017 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.144970    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:53.145017 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.145009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:53.145143 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.145050    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:53.145143 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.145066    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:44:53.145143 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.145101    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:54.145088336 +0000 UTC m=+32.872143453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:44:53.145143 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.145109    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:44:53.145143 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.145127    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:44:53.145343 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.145164    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:54.145149964 +0000 UTC m=+32.872205103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:44:53.145343 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.145177    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:54.145171882 +0000 UTC m=+32.872227000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:44:53.156584 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.156498    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7kq9h"]
Oct 30 10:44:53.158529 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:53.158505    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88e5148_03db_43f8_a46f_107d8d42af0f.slice/crio-d9916168be19512c0b0283a7a241769e5767ec2a932035458d16749cb8db9656 WatchSource:0}: Error finding container d9916168be19512c0b0283a7a241769e5767ec2a932035458d16749cb8db9656: Status 404 returned error can't find the container with id d9916168be19512c0b0283a7a241769e5767ec2a932035458d16749cb8db9656
Oct 30 10:44:53.246414 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.246359    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:53.246500 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.246448    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:44:53.246500 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.246490    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:44:54.24647973 +0000 UTC m=+32.973534864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:44:53.800106 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.797648    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:53.800106 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.797711    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:53.800106 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.797726    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:53.812820 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.812767    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 10:44:53.812939 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.812919    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 10:44:53.812992 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.812978    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-vmnnx\""
Oct 30 10:44:53.813055 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.813046    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-55bss\""
Oct 30 10:44:53.813105 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.813081    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c9pg5\""
Oct 30 10:44:53.954390 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:53.953306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:53.954390 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.953963    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:53.954390 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.953980    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:44:53.954390 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:53.954049    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:55.95401132 +0000 UTC m=+34.681066443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:44:54.045277 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.045197    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz" event={"ID":"fa764bd7-2feb-4a11-880e-70118d1ac639","Type":"ContainerStarted","Data":"2a552d464a9d49f9ac1dd857d287882eb8c4f367f58da3022c858f796ab898f9"}
Oct 30 10:44:54.047060 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.046971    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn" event={"ID":"b8b2a817-e3d9-4361-98cd-c0a40c93b84d","Type":"ContainerStarted","Data":"eb0076369a88562ee31eb074acf596cb34012dff875fdf385b0709be67695468"}
Oct 30 10:44:54.050879 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.050389    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-r9ns7" event={"ID":"e3cc17ea-0cec-4a71-882d-2df0457dd957","Type":"ContainerStarted","Data":"38c87eccc34b172a2a65ac4d1045bd8bd303314d4065d2c3350363f30c160136"}
Oct 30 10:44:54.052358 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.052333    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xfp2x" event={"ID":"418cb483-75bb-49b0-9c67-f3b70c26c2b4","Type":"ContainerStarted","Data":"dede0050d4d591ac08b32ed60b53b8f94c2ff8107ba8c4b25b5fafc77bf6a1d2"}
Oct 30 10:44:54.053937 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.053915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:54.054045 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.054003    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:54.054100 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.054080    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:54.054747 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.054227    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:54.054747 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.054287    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls podName:1cbcd9b8-ce20-494c-9a66-475272f2a561 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:56.054270627 +0000 UTC m=+34.781325751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lw745" (UID: "1cbcd9b8-ce20-494c-9a66-475272f2a561") : secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:54.054747 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.054629    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:44:56.054615095 +0000 UTC m=+34.781670231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : configmap references non-existent config key: service-ca.crt
Oct 30 10:44:54.054747 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.054691    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:44:54.054747 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.054721    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:44:56.054711958 +0000 UTC m=+34.781767079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:44:54.055924 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.055861    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6" event={"ID":"04a41ddb-fcb6-49bf-8468-deccd08a460c","Type":"ContainerStarted","Data":"a166dcbe1caca639bf7ee98a7a9d253f88e1d58d3204bd32d6e5b0aea39db67c"}
Oct 30 10:44:54.058546 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.058142    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8pgqv" event={"ID":"f384be3a-0f13-4b0a-ba1b-d63d2095b9d2","Type":"ContainerStarted","Data":"40f31ed4001313f6f9c0a53b69498ee256013bbcef8eb164574d0c7b443f9960"}
Oct 30 10:44:54.060377 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.060321    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7kq9h" event={"ID":"d88e5148-03db-43f8-a46f-107d8d42af0f","Type":"ContainerStarted","Data":"d9916168be19512c0b0283a7a241769e5767ec2a932035458d16749cb8db9656"}
Oct 30 10:44:54.089307 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.089217    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-r9ns7" podStartSLOduration=500.089201905 podStartE2EDuration="8m20.089201905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:54.088398899 +0000 UTC m=+32.815454040" watchObservedRunningTime="2026-10-30 10:44:54.089201905 +0000 UTC m=+32.816257046"
Oct 30 10:44:54.155142 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.155114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:54.155263 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.155213    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:54.155317 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.155284    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:54.155779 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.155758    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:54.155779 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.155779    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:44:54.155924 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.155824    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:56.155807085 +0000 UTC m=+34.882862218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:44:54.156205 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.156186    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:44:54.156278 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.156238    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:56.156224738 +0000 UTC m=+34.883279871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:44:54.156608 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.156592    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:44:54.156693 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.156643    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:56.156626429 +0000 UTC m=+34.883681564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:44:54.167175 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.167130    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-xfp2x" podStartSLOduration=500.167100711 podStartE2EDuration="8m20.167100711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:54.1665871 +0000 UTC m=+32.893642240" watchObservedRunningTime="2026-10-30 10:44:54.167100711 +0000 UTC m=+32.894155853"
Oct 30 10:44:54.167557 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.167510    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-8pgqv" podStartSLOduration=500.167503259 podStartE2EDuration="8m20.167503259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:54.12923153 +0000 UTC m=+32.856286671" watchObservedRunningTime="2026-10-30 10:44:54.167503259 +0000 UTC m=+32.894558416"
Oct 30 10:44:54.256737 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:54.256710    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:54.256861 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.256841    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:44:54.256927 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:54.256891    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:44:56.256875784 +0000 UTC m=+34.983930905 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:44:55.584800 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.584708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:55.584800 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.584780    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:44:55.585243 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:55.585111    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 10:44:55.585243 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:55.585175    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:45:27.58515574 +0000 UTC m=+66.312210861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : secret "metrics-daemon-secret" not found
Oct 30 10:44:55.592145 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.592117    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:55.625663 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.625637    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:44:55.686330 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.686296    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:55.693044 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.689769    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:55.946477 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.946404    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lkmz6"]
Oct 30 10:44:55.977759 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.977734    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:44:55.991083 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:55.990473    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:44:55.991083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:55.990696    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:55.991083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:55.990712    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:44:55.991083 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:55.990761    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:59.990743723 +0000 UTC m=+38.717798857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:44:56.082280 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.082229    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lkmz6" event={"ID":"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f","Type":"ContainerStarted","Data":"0a799cd7367e63ae9838196bf3ce1143b45d0cf9f88c5f14f27e6a79ac305a23"}
Oct 30 10:44:56.085818 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.084963    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn" event={"ID":"b8b2a817-e3d9-4361-98cd-c0a40c93b84d","Type":"ContainerStarted","Data":"4cb0119d103459a0a57296db094b7408802ee8548606acc0df393fe3314a2b7e"}
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.091293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.091372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.091426    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.091477    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.091455275 +0000 UTC m=+38.818510414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : configmap references non-existent config key: service-ca.crt
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.091540    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.091588    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls podName:1cbcd9b8-ce20-494c-9a66-475272f2a561 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.091573572 +0000 UTC m=+38.818628694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lw745" (UID: "1cbcd9b8-ce20-494c-9a66-475272f2a561") : secret "cluster-monitoring-operator-tls" not found
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.091677    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:44:56.091759 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.091711    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.091698585 +0000 UTC m=+38.818753727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:44:56.125413 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.125362    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn" podStartSLOduration=470.024901033 podStartE2EDuration="7m52.125350403s" totalImagesPullingTime="2.10044937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:56.124578931 +0000 UTC m=+34.851634071" watchObservedRunningTime="2026-10-30 10:44:56.125350403 +0000 UTC m=+34.852405542"
Oct 30 10:44:56.192363 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.192301    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:44:56.192508 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.192448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:44:56.192566 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.192509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:44:56.192661 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.192644    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:44:56.192721 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.192708    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.19269012 +0000 UTC m=+38.919745255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:44:56.193064 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.193040    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:44:56.193184 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.193097    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.193081649 +0000 UTC m=+38.920136849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:44:56.193540 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.193520    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:44:56.193540 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.193542    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:44:56.193677 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.193585    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.193570761 +0000 UTC m=+38.920625883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:44:56.293743 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.293673    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:44:56.293949 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.293933    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:44:56.294039 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:56.293993    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.293976542 +0000 UTC m=+39.021031723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:44:56.330509 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:56.330478    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ccnv5"]
Oct 30 10:44:56.337124 ip-10-0-140-230 kubenswrapper[2584]: W1030 10:44:56.337097    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167b7d19_70a6_43c4_a2c2_4386a574d3c3.slice/crio-b4b5be8fc34555c4ed21f6c826b523b123497f230152cedc0c172a407058e849 WatchSource:0}: Error finding container b4b5be8fc34555c4ed21f6c826b523b123497f230152cedc0c172a407058e849: Status 404 returned error can't find the container with id b4b5be8fc34555c4ed21f6c826b523b123497f230152cedc0c172a407058e849
Oct 30 10:44:57.091556 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:57.091494    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ccnv5" event={"ID":"167b7d19-70a6-43c4-a2c2-4386a574d3c3","Type":"ContainerStarted","Data":"b4b5be8fc34555c4ed21f6c826b523b123497f230152cedc0c172a407058e849"}
Oct 30 10:44:58.103992 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.103962    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/0.log"
Oct 30 10:44:58.104392 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.104009    2584 generic.go:356] "Generic (PLEG): container finished" podID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" containerID="9c98017259e714c0190079d93f72f2def6bbb48bddf3e37771638c86ae2850af" exitCode=255
Oct 30 10:44:58.104452 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.104403    2584 scope.go:122] "RemoveContainer" containerID="9c98017259e714c0190079d93f72f2def6bbb48bddf3e37771638c86ae2850af"
Oct 30 10:44:58.104646 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.104625    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" event={"ID":"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a","Type":"ContainerDied","Data":"9c98017259e714c0190079d93f72f2def6bbb48bddf3e37771638c86ae2850af"}
Oct 30 10:44:58.109525 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.108089    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst" event={"ID":"7333057e-e1db-4970-bf86-eede9e94c06a","Type":"ContainerStarted","Data":"e9bd4aa37b4e865374d425a95b2a451b0b61a01d995b5c0712722a05daeed3d0"}
Oct 30 10:44:58.112079 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.111942    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-v69fn" event={"ID":"9e199587-19b7-4a4b-bdd0-761b7a6e475f","Type":"ContainerStarted","Data":"c72186ae410aa7a8ddf608b6ce5fa204dad865cf52b9b90bd9ac91862be54c05"}
Oct 30 10:44:58.191198 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.191142    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-v69fn" podStartSLOduration=470.48546083 podStartE2EDuration="7m55.191123763s" totalImagesPullingTime="4.705662933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:58.188436657 +0000 UTC m=+36.915491798" watchObservedRunningTime="2026-10-30 10:44:58.191123763 +0000 UTC m=+36.918178904"
Oct 30 10:44:58.503017 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.502928    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst" podStartSLOduration=470.822002969 podStartE2EDuration="7m55.502909229s" totalImagesPullingTime="4.68090626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:58.228256711 +0000 UTC m=+36.955311855" watchObservedRunningTime="2026-10-30 10:44:58.502909229 +0000 UTC m=+37.229964368"
Oct 30 10:44:58.503219 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.503116    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2kxlm"]
Oct 30 10:44:58.517536 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.517510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.526554 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.526494    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2kxlm"]
Oct 30 10:44:58.533255 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.533095    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 10:44:58.533331 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.533286    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sssm5\""
Oct 30 10:44:58.614739 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.614713    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 10:44:58.618584 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.618562    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.618689 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.618621    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpl2q\" (UniqueName: \"kubernetes.io/projected/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-api-access-lpl2q\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.618689 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.618651    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ca9bba-a38c-48af-9a31-f71bad8bb876-data-volume\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.618689 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.618676    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.618793 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.618745    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07ca9bba-a38c-48af-9a31-f71bad8bb876-crio-socket\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.719624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07ca9bba-a38c-48af-9a31-f71bad8bb876-crio-socket\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.719744    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.719807    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpl2q\" (UniqueName: \"kubernetes.io/projected/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-api-access-lpl2q\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.719834    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ca9bba-a38c-48af-9a31-f71bad8bb876-data-volume\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.719854    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:58.719971    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:58.720014    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:59.220001607 +0000 UTC m=+37.947056725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.720089    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07ca9bba-a38c-48af-9a31-f71bad8bb876-crio-socket\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:58.720126    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:58.720150    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:44:59.220142463 +0000 UTC m=+37.947197581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : configmap "kube-rbac-proxy" not found
Oct 30 10:44:58.720784 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.720553    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ca9bba-a38c-48af-9a31-f71bad8bb876-data-volume\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:58.744262 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:58.744219    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpl2q\" (UniqueName: \"kubernetes.io/projected/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-api-access-lpl2q\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:59.119477 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.119436    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6" event={"ID":"04a41ddb-fcb6-49bf-8468-deccd08a460c","Type":"ContainerStarted","Data":"8656aa12033f59b56f04ce7d16a7cf56e64cd119eab5c6223ddad65ad0da292c"}
Oct 30 10:44:59.122792 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.122708    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/1.log"
Oct 30 10:44:59.123262 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.123241    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/0.log"
Oct 30 10:44:59.123386 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.123278    2584 generic.go:356] "Generic (PLEG): container finished" podID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" containerID="f8912bf9c29e99061121aba6a3b280560ad379cf25a59577044eb706d3d8e5f9" exitCode=255
Oct 30 10:44:59.123386 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.123352    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" event={"ID":"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a","Type":"ContainerDied","Data":"f8912bf9c29e99061121aba6a3b280560ad379cf25a59577044eb706d3d8e5f9"}
Oct 30 10:44:59.123460 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.123397    2584 scope.go:122] "RemoveContainer" containerID="9c98017259e714c0190079d93f72f2def6bbb48bddf3e37771638c86ae2850af"
Oct 30 10:44:59.124056 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.123748    2584 scope.go:122] "RemoveContainer" containerID="f8912bf9c29e99061121aba6a3b280560ad379cf25a59577044eb706d3d8e5f9"
Oct 30 10:44:59.124056 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:59.123943    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2hmr5_openshift-console-operator(909137f2-6cf7-4fc8-8bcd-b0620b87aa5a)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" podUID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a"
Oct 30 10:44:59.127098 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.126782    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7kq9h" event={"ID":"d88e5148-03db-43f8-a46f-107d8d42af0f","Type":"ContainerStarted","Data":"96d6cfc7e319e4a65bc3dd3d084dc1c88d67bba7eed2fbc9ea6e36ee3b0cc94c"}
Oct 30 10:44:59.167148 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.167084    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6" podStartSLOduration=470.118314729 podStartE2EDuration="7m55.167069718s" totalImagesPullingTime="5.048754989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:59.166716765 +0000 UTC m=+37.893771906" watchObservedRunningTime="2026-10-30 10:44:59.167069718 +0000 UTC m=+37.894124860"
Oct 30 10:44:59.223929 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.223899    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:59.225709 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:59.225172    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 10:44:59.225912 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:59.225879    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.225856126 +0000 UTC m=+38.952911248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : configmap "kube-rbac-proxy" not found
Oct 30 10:44:59.225991 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.225943    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:44:59.226907 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:59.226347    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:44:59.226907 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:44:59.226440    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:00.226415376 +0000 UTC m=+38.953470509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:44:59.250855 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:44:59.250809    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7kq9h" podStartSLOduration=1.8813047489999999 podStartE2EDuration="7.25079717s" totalImagesPullingTime="5.369492421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:44:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:44:59.249555396 +0000 UTC m=+37.976610539" watchObservedRunningTime="2026-10-30 10:44:59.25079717 +0000 UTC m=+37.977852311"
Oct 30 10:45:00.034440 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.034409    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:00.034982 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.034963    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:00.035043 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.034986    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:45:00.035100 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.035062    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.035042763 +0000 UTC m=+46.762097888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:45:00.135089 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.135047    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:00.135498 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.135127    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:00.135498 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.135261    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:45:00.135498 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.135350    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.135335345 +0000 UTC m=+46.862390464 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:45:00.135498 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.135364    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.135358597 +0000 UTC m=+46.862413714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : configmap references non-existent config key: service-ca.crt
Oct 30 10:45:00.135498 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.135266    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:00.135498 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.135405    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 10:45:00.135498 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.135444    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls podName:1cbcd9b8-ce20-494c-9a66-475272f2a561 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.135429746 +0000 UTC m=+46.862484864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lw745" (UID: "1cbcd9b8-ce20-494c-9a66-475272f2a561") : secret "cluster-monitoring-operator-tls" not found
Oct 30 10:45:00.137079 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.137013    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ccnv5" event={"ID":"167b7d19-70a6-43c4-a2c2-4386a574d3c3","Type":"ContainerStarted","Data":"effd03837520561231ce2edea7c1d7b6691a60eea66e2c09b473d49143079ab0"}
Oct 30 10:45:00.137197 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.137132    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:45:00.138777 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.138759    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/1.log"
Oct 30 10:45:00.139091 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.139070    2584 scope.go:122] "RemoveContainer" containerID="f8912bf9c29e99061121aba6a3b280560ad379cf25a59577044eb706d3d8e5f9"
Oct 30 10:45:00.139266 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.139242    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2hmr5_openshift-console-operator(909137f2-6cf7-4fc8-8bcd-b0620b87aa5a)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" podUID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a"
Oct 30 10:45:00.140367 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.140324    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz" event={"ID":"fa764bd7-2feb-4a11-880e-70118d1ac639","Type":"ContainerStarted","Data":"2dc7930151a6fc391085768ebabd0b24cf8fc67c0a2bd890a37b25b18c3edfe9"}
Oct 30 10:45:00.177970 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.177889    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ccnv5" podStartSLOduration=33.598094826 podStartE2EDuration="37.177875158s" totalImagesPullingTime="3.579780332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:45:00.177470151 +0000 UTC m=+38.904525293" watchObservedRunningTime="2026-10-30 10:45:00.177875158 +0000 UTC m=+38.904930302"
Oct 30 10:45:00.217912 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.217858    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz" podStartSLOduration=469.820159844 podStartE2EDuration="7m56.21784575s" totalImagesPullingTime="6.397685906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:45:00.216130288 +0000 UTC m=+38.943185427" watchObservedRunningTime="2026-10-30 10:45:00.21784575 +0000 UTC m=+38.944900890"
Oct 30 10:45:00.235891 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.235865    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:00.235991 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.235905    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:00.235991 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.235962    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:00.236133 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.235984    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 10:45:00.236133 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.236037    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:00.236133 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236050    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:02.236017453 +0000 UTC m=+40.963072571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : configmap "kube-rbac-proxy" not found
Oct 30 10:45:00.236133 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236071    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:45:00.236133 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236101    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:00.236133 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236113    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:45:00.236133 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236123    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:00.236457 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236125    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.236111163 +0000 UTC m=+46.963166294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:45:00.236457 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236193    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.236179972 +0000 UTC m=+46.963235099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:45:00.236457 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236208    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:02.236201044 +0000 UTC m=+40.963256163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:00.236457 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.236250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:00.236457 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236356    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:45:00.236457 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.236392    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.236383575 +0000 UTC m=+46.963438707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:45:00.337642 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:00.337609    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:00.337979 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.337954    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:45:00.338102 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:00.338087    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:45:08.338065134 +0000 UTC m=+47.065120257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:45:02.251880 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:02.251847    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:02.252282 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:02.251906    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:02.252282 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:02.251972    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 10:45:02.252282 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:02.252057    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:06.252040337 +0000 UTC m=+44.979095455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : configmap "kube-rbac-proxy" not found
Oct 30 10:45:02.252282 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:02.252096    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:02.252282 ip-10-0-140-230 kubenswrapper[2584]: E1030 10:45:02.252150    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:06.252137927 +0000 UTC m=+44.979193067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:02.549106 ip-10-0-140-230 kubenswrapper[2584]: I1030 10:45:02.548959    2584 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 10:45:02.549170 ip-10-0-140-230 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 10:45:02.565360 ip-10-0-140-230 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 10:45:02.565581 ip-10-0-140-230 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 10:45:02.565637 ip-10-0-140-230 systemd[1]: kubelet.service: Consumed 2.775s CPU time.
Oct 30 10:45:02.579482 ip-10-0-140-230 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 10:45:02.661643 ip-10-0-140-230 kubenswrapper[4849]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 10:45:02.661643 ip-10-0-140-230 kubenswrapper[4849]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 10:45:02.661643 ip-10-0-140-230 kubenswrapper[4849]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 10:45:02.661643 ip-10-0-140-230 kubenswrapper[4849]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 10:45:02.663906 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663829    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.663906 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663907    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663915    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663920    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663925    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663929    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663933    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663937    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663941    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663946    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663950    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663954    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663958    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663962    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663966    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663970    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663975    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663980    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663984    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663988    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.664095 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663991    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663995    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.663999    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664003    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664007    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664013    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664034    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664039    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664045    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664050    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664054    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664059    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664064    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664070    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664074    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664079    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664083    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664087    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664091    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664095    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.664915 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664099    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664104    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664108    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664112    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664116    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664120    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664124    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664128    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664133    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664138    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664142    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664146    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664150    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664154    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664159    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664163    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664168    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664172    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664176    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664181    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.665822 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664186    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664190    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664194    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664198    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664203    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664207    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664211    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664216    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664220    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664224    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664229    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664233    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664237    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664241    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664246    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664250    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664254    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664260    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664264    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.666762 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664269    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664273    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664277    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664282    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664286    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664290    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664295    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664298    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664303    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664307    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664312    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664316    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664320    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664325    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664329    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664333    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664338    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664342    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664346    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664350    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.667540 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664354    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664359    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664364    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664368    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664373    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664377    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.664383    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665653    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665664    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665668    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665673    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665677    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665682    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665686    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665690    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665697    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665702    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665707    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665711    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.668391 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665716    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665720    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665724    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665729    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665733    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665737    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665741    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665745    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665750    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665754    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665759    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665763    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665767    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665772    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665776    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665780    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665784    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665788    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665793    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.668875 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665797    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665801    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665805    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665810    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665814    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665818    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665827    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665833    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665838    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665843    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665848    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665853    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665858    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665862    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665869    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665875    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665880    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665884    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665888    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.669393 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665893    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665897    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665901    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665906    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665910    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665914    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665918    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665922    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665927    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665932    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665936    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665940    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665944    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665949    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665953    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665957    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665962    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665966    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665971    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.669877 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665977    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665981    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665985    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665989    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665994    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.665998    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666011    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666015    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666035    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666040    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666044    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666048    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666053    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666057    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666061    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666065    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666069    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666074    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666078    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666082    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.670401 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666087    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666091    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666096    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666100    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666105    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666109    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666113    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666119    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666124    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666128    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666133    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666137    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666141    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666146    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666150    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666154    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.666158    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668005    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668053    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668060    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.670902 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668072    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668077    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668082    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668086    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668091    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668096    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668101    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668105    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668110    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668114    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668118    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668123    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668128    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668137    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668141    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668147    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668151    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668156    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668161    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.671404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668166    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668170    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668175    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668180    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668184    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668189    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668193    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668203    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668208    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668212    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668217    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668222    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668260    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668266    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668271    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668275    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668279    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668284    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668289    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.671871 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668298    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668303    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668307    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668316    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668334    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668477    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668487    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668493    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668496    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668499    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668502    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668506    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668509    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668512    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668515    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668518    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668521    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668523    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668526    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668529    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.672621 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668532    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668535    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668537    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668540    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668544    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668546    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668549    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668552    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668555    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668557    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668560    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668563    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668565    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668568    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668571    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668573    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668578    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668583    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668586    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.673547 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668589    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668591    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668597    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668599    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668602    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668605    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668608    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668611    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668614    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668616    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668622    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668625    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668628    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668631    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668634    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668637    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668640    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668643    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668645    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668648    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.674403 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668650    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668654    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668656    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668662    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668667    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.668670    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668773    4849 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668780    4849 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668787    4849 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668792    4849 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668797    4849 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668800    4849 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668807    4849 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668811    4849 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668814    4849 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668817    4849 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668821    4849 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668825    4849 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668828    4849 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668831    4849 flags.go:64] FLAG: --cgroup-root=""
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668834    4849 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668837    4849 flags.go:64] FLAG: --client-ca-file=""
Oct 30 10:45:02.675329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668842    4849 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668845    4849 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668850    4849 flags.go:64] FLAG: --cluster-domain=""
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668853    4849 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668856    4849 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668859    4849 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668863    4849 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668867    4849 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668871    4849 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668874    4849 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668877    4849 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668881    4849 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668884    4849 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668887    4849 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668891    4849 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668894    4849 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668898    4849 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668901    4849 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668904    4849 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668907    4849 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668910    4849 flags.go:64] FLAG: --enable-server="true"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668913    4849 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668918    4849 flags.go:64] FLAG: --event-burst="100"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668921    4849 flags.go:64] FLAG: --event-qps="50"
Oct 30 10:45:02.676311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668925    4849 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668928    4849 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668930    4849 flags.go:64] FLAG: --eviction-hard=""
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668935    4849 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668939    4849 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668942    4849 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668945    4849 flags.go:64] FLAG: --eviction-soft=""
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668949    4849 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668953    4849 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668956    4849 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668959    4849 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668962    4849 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668965    4849 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668968    4849 flags.go:64] FLAG: --feature-gates=""
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668972    4849 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668976    4849 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668979    4849 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668982    4849 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668985    4849 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668988    4849 flags.go:64] FLAG: --help="false"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668991    4849 flags.go:64] FLAG: --hostname-override="ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668994    4849 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.668997    4849 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 10:45:02.677462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669000    4849 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669004    4849 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669007    4849 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669010    4849 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669013    4849 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669016    4849 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669033    4849 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669038    4849 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669043    4849 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669047    4849 flags.go:64] FLAG: --kube-reserved=""
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669051    4849 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669056    4849 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669059    4849 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669063    4849 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669066    4849 flags.go:64] FLAG: --lock-file=""
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669069    4849 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669072    4849 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669080    4849 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669086    4849 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669089    4849 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669092    4849 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669095    4849 flags.go:64] FLAG: --logging-format="text"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669098    4849 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669101    4849 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 10:45:02.678530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669104    4849 flags.go:64] FLAG: --manifest-url=""
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669107    4849 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669112    4849 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669115    4849 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669119    4849 flags.go:64] FLAG: --max-pods="110"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669122    4849 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669125    4849 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669128    4849 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669131    4849 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669134    4849 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669137    4849 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669140    4849 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669147    4849 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669151    4849 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669153    4849 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669157    4849 flags.go:64] FLAG: --pod-cidr=""
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669160    4849 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669163    4849 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669166    4849 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669171    4849 flags.go:64] FLAG: --port="10250"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669174    4849 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669177    4849 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04876cbcfe57cb72c"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669181    4849 flags.go:64] FLAG: --qos-reserved=""
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669184    4849 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669187    4849 flags.go:64] FLAG: --register-node="true"
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669191    4849 flags.go:64] FLAG: --register-with-taints=""
Oct 30 10:45:02.679644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669195    4849 flags.go:64] FLAG: --registry-burst="10"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669198    4849 flags.go:64] FLAG: --registry-qps="5"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669201    4849 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669204    4849 flags.go:64] FLAG: --reserved-memory=""
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669208    4849 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669211    4849 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669214    4849 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669217    4849 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669220    4849 flags.go:64] FLAG: --runonce="false"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669223    4849 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669227    4849 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669230    4849 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669232    4849 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669235    4849 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669238    4849 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669242    4849 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669245    4849 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669247    4849 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669250    4849 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669254    4849 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669256    4849 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669260    4849 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669263    4849 flags.go:64] FLAG: --system-cgroups=""
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669265    4849 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669272    4849 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 10:45:02.680772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669278    4849 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669281    4849 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669285    4849 flags.go:64] FLAG: --tls-min-version=""
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669288    4849 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669291    4849 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669295    4849 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669299    4849 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669302    4849 flags.go:64] FLAG: --v="2"
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669306    4849 flags.go:64] FLAG: --version="false"
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669310    4849 flags.go:64] FLAG: --vmodule=""
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669314    4849 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669317    4849 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669465    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669469    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669472    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669475    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669478    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669481    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669484    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669486    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669489    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669492    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.681829 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669494    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669497    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669499    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669502    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669505    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669507    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669510    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669512    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669515    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669518    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669522    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669525    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669527    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669530    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669532    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669535    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669539    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669541    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669544    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.682514 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669547    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669549    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669552    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669555    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669557    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669560    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669562    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669566    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669570    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669573    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669576    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669579    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669582    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669585    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669587    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669590    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669593    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669595    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669598    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.683052 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669601    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669603    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669606    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669608    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669612    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669615    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669618    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669621    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669623    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669626    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669629    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669632    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669634    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669637    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669640    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669642    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669645    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669648    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669650    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669653    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.683518 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669655    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669658    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669661    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669664    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669666    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669669    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669673    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669676    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669679    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669682    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669685    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669687    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669690    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669692    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669695    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669698    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669703    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669705    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669708    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.684029 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669710    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669713    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669715    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669718    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669723    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669725    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669728    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669731    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669733    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669736    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669738    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669741    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669743    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669746    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669748    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669751    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669753    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669756    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.684684 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669759    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.669785    4849 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669822    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669826    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669829    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669832    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669836    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669840    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669843    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669846    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669850    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669854    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669859    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669863    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669866    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.685315 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669869    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669872    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669874    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669877    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669880    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669883    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669886    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669889    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669891    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669894    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669897    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669899    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669902    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669904    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669907    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669909    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669912    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669915    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669917    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669920    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.685948 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669922    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669925    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669927    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669930    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669934    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669936    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669939    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669942    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669944    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669948    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669950    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669953    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669955    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669958    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669960    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669963    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669965    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669968    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669971    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669973    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.686817 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669976    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669978    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669981    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669983    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669986    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669988    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669991    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669993    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669996    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.669998    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670001    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670003    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670006    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670008    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670011    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670014    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670040    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670044    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670048    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.687664 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670052    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670056    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670059    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670061    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670065    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670068    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670071    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670074    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670076    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670079    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670081    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670084    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670088    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670090    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670093    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670095    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670098    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670100    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670103    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.688477 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670105    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670108    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670110    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670113    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670116    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670119    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670122    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670125    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670128    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670131    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670136    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670138    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670141    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670143    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.670146    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.674943    4849 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.674961    4849 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.674984    4849 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.674992    4849 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675093    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.689290 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675102    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675108    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675112    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675117    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675122    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675126    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675130    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675135    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675139    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675144    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675148    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675152    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675156    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675161    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675165    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675169    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675174    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675178    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675182    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675187    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.689796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675191    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675195    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675200    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675204    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675208    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675213    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675217    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675221    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675227    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675231    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675238    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675245    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675250    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675255    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675259    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675264    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675268    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675272    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675277    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675281    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.690344 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675285    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675290    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675294    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675299    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675303    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675307    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675311    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675316    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675320    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675324    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675329    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675333    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675337    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675341    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675345    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675349    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675353    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675357    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675362    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675366    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.690838 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675371    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675376    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675380    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675384    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675389    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675393    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675397    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675401    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675405    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675409    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675413    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675418    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675423    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675427    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675431    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675436    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675440    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675444    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675449    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.691329 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675453    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675458    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675462    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675467    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675471    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675475    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675482    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675487    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675492    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675496    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675501    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675505    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675510    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675514    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675518    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675523    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675527    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675532    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675536    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.691774 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675540    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675544    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675548    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675553    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675558    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675562    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675566    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.675606    4849 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675886    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675895    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675899    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675905    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675910    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675914    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675919    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.692256 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675923    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675930    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675936    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675940    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675945    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675950    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675955    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675959    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675964    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675968    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675973    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675977    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675981    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675985    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675991    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.675995    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676000    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676004    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676009    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.692626 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676013    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676017    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676037    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676042    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676046    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676050    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676055    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676059    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676063    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676068    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676072    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676076    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676081    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676085    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676089    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676094    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676098    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676102    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676107    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676111    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.693100 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676115    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676119    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676124    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676128    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676134    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676139    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676144    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676149    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676154    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676158    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676165    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676169    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676174    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676178    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676182    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676186    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676191    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676195    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676199    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.693582 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676204    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676208    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676212    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676216    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676220    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676224    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676229    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676233    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676238    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676242    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676250    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676254    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676259    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676263    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676267    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676273    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676277    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676281    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676286    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.694114 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676291    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676295    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676299    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676304    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676314    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676319    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676323    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676327    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676331    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676335    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676339    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676343    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676348    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676352    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676357    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676361    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676365    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676369    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676373    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676378    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.694590 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676382    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676387    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.676428    4849 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676492    4849 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676501    4849 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676507    4849 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676511    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676517    4849 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676524    4849 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676531    4849 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676536    4849 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676541    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676546    4849 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676551    4849 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676556    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 10:45:02.695143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676560    4849 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676568    4849 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676572    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676577    4849 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676581    4849 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676585    4849 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676590    4849 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676594    4849 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676598    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676603    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676608    4849 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676612    4849 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676616    4849 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676621    4849 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676626    4849 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676630    4849 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676634    4849 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676639    4849 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676643    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676647    4849 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 10:45:02.695564 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676651    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676657    4849 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676662    4849 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676666    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676670    4849 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676674    4849 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676679    4849 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676683    4849 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676687    4849 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676691    4849 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676696    4849 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676700    4849 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676705    4849 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676711    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676716    4849 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676722    4849 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676728    4849 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676733    4849 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676738    4849 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676742    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 10:45:02.696058 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676748    4849 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676753    4849 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676757    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676761    4849 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676766    4849 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676770    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676774    4849 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676779    4849 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676783    4849 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676787    4849 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676792    4849 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676796    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676801    4849 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676807    4849 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676811    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676815    4849 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676820    4849 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676824    4849 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676829    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 10:45:02.696586 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676833    4849 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676837    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676841    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676846    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676850    4849 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676854    4849 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676861    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676865    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676870    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676874    4849 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676878    4849 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676883    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676887    4849 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676891    4849 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676895    4849 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676900    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676904    4849 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676909    4849 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676913    4849 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676917    4849 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 10:45:02.697112 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676922    4849 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676926    4849 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676930    4849 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676934    4849 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676939    4849 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676944    4849 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676952    4849 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676956    4849 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676961    4849 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676965    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676969    4849 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676974    4849 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676978    4849 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676983    4849 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:02.676987    4849 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.677111    4849 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.678636    4849 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.678704    4849 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 10:45:02.697610 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.678977    4849 server.go:1014] "Starting client certificate rotation"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.679146    4849 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.679192    4849 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 10:39:22 +0000 UTC" deadline="2028-04-30 00:54:43.239070473 +0000 UTC"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.679216    4849 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13142h9m40.559855668s"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.679722    4849 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.681342    4849 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.682531    4849 log.go:25] "Validated CRI v1 runtime API"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.684150    4849 log.go:25] "Validated CRI v1 image API"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.684848    4849 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 10:45:02.698096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.687451    4849 fs.go:135] Filesystem UUIDs: map[75d3f1e6-0c83-4693-9137-379361e5670f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dae90d51-1a36-46b8-b6ba-bc0dee7f10a6:/dev/nvme0n1p4]
Oct 30 10:45:02.698338 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.687477    4849 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0486f6c8dac9a33be956b33582bcd1aceb4e726edee6f89f4d452115f4e3126e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0486f6c8dac9a33be956b33582bcd1aceb4e726edee6f89f4d452115f4e3126e/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0a799cd7367e63ae9838196bf3ce1143b45d0cf9f88c5f14f27e6a79ac305a23/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0a799cd7367e63ae9838196bf3ce1143b45d0cf9f88c5f14f27e6a79ac305a23/userdata/shm major:0 minor:293 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/28555fb462459f87e6b3be52779d112f84230173a0f43e6b692179bc23a35d6d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/28555fb462459f87e6b3be52779d112f84230173a0f43e6b692179bc23a35d6d/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2a552d464a9d49f9ac1dd857d287882eb8c4f367f58da3022c858f796ab898f9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2a552d464a9d49f9ac1dd857d287882eb8c4f367f58da3022c858f796ab898f9/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3893e0286e812cd52687e6ac8f6a019efa64c80badf964f45f0c9b6a304c9fbd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3893e0286e812cd52687e6ac8f6a019efa64c80badf964f45f0c9b6a304c9fbd/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/56a24866a1d635ce8c8a021cc89c4208c214acf6ec65ba4d59ac8ca3051f4a7a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/56a24866a1d635ce8c8a021cc89c4208c214acf6ec65ba4d59ac8ca3051f4a7a/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/685fca13f89345449294586b150e424b00aab4055adb373a7e75d1cde410e851/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/685fca13f89345449294586b150e424b00aab4055adb373a7e75d1cde410e851/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74498fd8124d764c596898c75bc2ba32a62d0762c83634f5b5295bcdcfa6b10e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74498fd8124d764c596898c75bc2ba32a62d0762c83634f5b5295bcdcfa6b10e/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/78c49b901c1a17ecb9be8a9af3c2b860357d0cd0e2b27b036f9b41b03a23cf28/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/78c49b901c1a17ecb9be8a9af3c2b860357d0cd0e2b27b036f9b41b03a23cf28/userdata/shm major:0 minor:212 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/91f94347ed9c3b23754ab3ea271fe0549d3f60a3ab0df7f4d1427868b99627db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/91f94347ed9c3b23754ab3ea271fe0549d3f60a3ab0df7f4d1427868b99627db/userdata/shm major:0 minor:222 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/997deae2ef22fb6b0f9c791a7454c5aa9821388b03fd76c40b717f3f95c4df0c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/997deae2ef22fb6b0f9c791a7454c5aa9821388b03fd76c40b717f3f95c4df0c/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a166dcbe1caca639bf7ee98a7a9d253f88e1d58d3204bd32d6e5b0aea39db67c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a166dcbe1caca639bf7ee98a7a9d253f88e1d58d3204bd32d6e5b0aea39db67c/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a4f499333b507c41f0d264032cdea35113c6b5032c791e83932ba287418a6ffe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a4f499333b507c41f0d264032cdea35113c6b5032c791e83932ba287418a6ffe/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/af02cc1967f17431e480c5a28b0f6de45431f63d9e431bba6c130b0956e3d0f9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/af02cc1967f17431e480c5a28b0f6de45431f63d9e431bba6c130b0956e3d0f9/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b4b5be8fc34555c4ed21f6c826b523b123497f230152cedc0c172a407058e849/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b4b5be8fc34555c4ed21f6c826b523b123497f230152cedc0c172a407058e849/userdata/shm major:0 minor:298 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d7436000ac04ff26222ea5a272d6bf8de3d44865f7d1705a41aebffd964ae6ee/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d7436000ac04ff26222ea5a272d6bf8de3d44865f7d1705a41aebffd964ae6ee/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d9916168be19512c0b0283a7a241769e5767ec2a932035458d16749cb8db9656/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d9916168be19512c0b0283a7a241769e5767ec2a932035458d16749cb8db9656/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb0076369a88562ee31eb074acf596cb34012dff875fdf385b0709be67695468/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb0076369a88562ee31eb074acf596cb34012dff875fdf385b0709be67695468/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef9d119ab837d6ffad02fb6eb0c44f4893af7ded3f78bfbb2903d34d72c6032c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef9d119ab837d6ffad02fb6eb0c44f4893af7ded3f78bfbb2903d34d72c6032c/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff0e1831cc43c694d6d09a2b2fbe9a14163550eee2d9fa8c5875c489c6b44861/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff0e1831cc43c694d6d09a2b2fbe9a14163550eee2d9fa8c5875c489c6b44861/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/04a41ddb-fcb6-49bf-8468-deccd08a460c/volumes/kubernetes.io~projected/kube-api-access-8v75s:{mountpoint:/var/lib/kubelet/pods/04a41ddb-fcb6-49bf-8468-deccd08a460c/volumes/kubernetes.io~projected/kube-api-access-8v75s major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/04a41ddb-fcb6-49bf-8468-deccd08a460c/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/04a41ddb-fcb6-49bf-8468-deccd08a460c/volumes/kubernetes.io~secret/serving-cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07ca9bba-a38c-48af-9a31-f71bad8bb876/volumes/kubernetes.io~projected/kube-api-access-lpl2q:{mountpoint:/var/lib/kubelet/pods/07ca9bba-a38c-48af-9a31-f71bad8bb876/volumes/kubernetes.io~projected/kube-api-access-lpl2q major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0df37afa-77d5-4bf7-bc83-0d7636453bda/volumes/kubernetes.io~projected/kube-api-access-qlsp5:{mountpoint:/var/lib/kubelet/pods/0df37afa-77d5-4bf7-bc83-0d7636453bda/volumes/kubernetes.io~projected/kube-api-access-qlsp5 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1114f0eb-6317-4004-88fa-55f60fc991a5/volumes/kubernetes.io~projected/kube-api-access-twjpp:{mountpoint:/var/lib/kubelet/pods/1114f0eb-6317-4004-88fa-55f60fc991a5/volumes/kubernetes.io~projected/kube-api-access-twjpp major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/167b7d19-70a6-43c4-a2c2-4386a574d3c3/volumes/kubernetes.io~projected/kube-api-access-b42qs:{mountpoint:/var/lib/kubelet/pods/167b7d19-70a6-43c4-a2c2-4386a574d3c3/volumes/kubernetes.io~projected/kube-api-access-b42qs major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~projected/kube-api-access-r2fpx:{mountpoint:/var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~projected/kube-api-access-r2fpx major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1cbcd9b8-ce20-494c-9a66-475272f2a561/volumes/kubernetes.io~projected/kube-api-access-hlkjh:{mountpoint:/var/lib/kubelet/pods/1cbcd9b8-ce20-494c-9a66-475272f2a561/volumes/kubernetes.io~projected/kube-api-access-hlkjh major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fba2151-22de-4a03-afcb-1d89b9c32a2b/volumes/kubernetes.io~projected/kube-api-access-qshr8:{mountpoint:/var/lib/kubelet/pods/2fba2151-22de-4a03-afcb-1d89b9c32a2b/volumes/kubernetes.io~projected/kube-api-access-qshr8 major:0 minor:242 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/418cb483-75bb-49b0-9c67-f3b70c26c2b4/volumes/kubernetes.io~projected/kube-api-access-hq49n:{mountpoint:/var/lib/kubelet/pods/418cb483-75bb-49b0-9c67-f3b70c26c2b4/volumes/kubernetes.io~projected/kube-api-access-hq49n major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~projected/kube-api-access-4hct8:{mountpoint:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~projected/kube-api-access-4hct8 major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7333057e-e1db-4970-bf86-eede9e94c06a/volumes/kubernetes.io~projected/kube-api-access-6ncc7:{mountpoint:/var/lib/kubelet/pods/7333057e-e1db-4970-bf86-eede9e94c06a/volumes/kubernetes.io~projected/kube-api-access-6ncc7 major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7333057e-e1db-4970-bf86-eede9e94c06a/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/7333057e-e1db-4970-bf86-eede9e94c06a/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7add3842-20f6-49eb-89ad-02078f7767ef/volumes/kubernetes.io~projected/kube-api-access-vtrmh:{mountpoint:/var/lib/kubelet/pods/7add3842-20f6-49eb-89ad-02078f7767ef/volumes/kubernetes.io~projected/kube-api-access-vtrmh major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/volumes/kubernetes.io~projected/kube-api-access-tvnqf:{mountpoint:/var/lib/kubelet/pods/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/volumes/kubernetes.io~projected/kube-api-access-tvnqf major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/volumes/kubernetes.io~secret/serving-cert major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/984f643d-51d1-4142-b244-f8f0980a055f/volumes/kubernetes.io~projected/kube-api-access-hxbzw:{mountpoint:/var/lib/kubelet/pods/984f643d-51d1-4142-b244-f8f0980a055f/volumes/kubernetes.io~projected/kube-api-access-hxbzw major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~projected/kube-api-access-54d74:{mountpoint:/var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~projected/kube-api-access-54d74 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~secret/default-certificate major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~secret/stats-auth major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e199587-19b7-4a4b-bdd0-761b7a6e475f/volumes/kubernetes.io~projected/kube-api-access-cqwn6:{mountpoint:/var/lib/kubelet/pods/9e199587-19b7-4a4b-bdd0-761b7a6e475f/volumes/kubernetes.io~projected/kube-api-access-cqwn6 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e199587-19b7-4a4b-bdd0-761b7a6e475f/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/9e199587-19b7-4a4b-bdd0-761b7a6e475f/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~projected/kube-api-access-6jnkk:{mountpoint:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~projected/kube-api-access-6jnkk major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a179c33d-aecd-4b72-b907-37f82a9aec15/volumes/kubernetes.io~projected/kube-api-access-qn5vl:{mountpoint:/var/lib/kubelet/pods/a179c33d-aecd-4b72-b907-37f82a9aec15/volumes/kubernetes.io~projected/kube-api-access-qn5vl major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8b2a817-e3d9-4361-98cd-c0a40c93b84d/volumes/kubernetes.io~projected/kube-api-access-z2hnd:{mountpoint:/var/lib/kubelet/pods/b8b2a817-e3d9-4361-98cd-c0a40c93b84d/volumes/kubernetes.io~projected/kube-api-access-z2hnd major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d88e5148-03db-43f8-a46f-107d8d42af0f/volumes/kubernetes.io~projected/kube-api-access-vq8tw:{mountpoint:/var/lib/kubelet/pods/d88e5148-03db-43f8-a46f-107d8d42af0f/volumes/kubernetes.io~projected/kube-api-access-vq8tw major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d88e5148-03db-43f8-a46f-107d8d42af0f/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d88e5148-03db-43f8-a46f-107d8d42af0f/volumes/kubernetes.io~secret/cert major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc78343b-907d-4222-b27d-7705a2a0f19e/volumes/kubernetes.io~projected/kube-api-access-kml79:{mountpoint:/var/lib/kubelet/pods/dc78343b-907d-4222-b27d-7705a2a0f19e/volumes/kubernetes.io~projected/kube-api-access-kml79 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ded181d7-f3c4-4324-946f-2770e8e3d0fd/volumes/kubernetes.io~projected/kube-api-access-758g5:{mountpoint:/var/lib/kubelet/pods/ded181d7-f3c4-4324-946f-2770e8e3d0fd/volumes/kubernetes.io~projected/kube-api-access-758g5 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e3cc17ea-0cec-4a71-882d-2df0457dd957/volumes/kubernetes.io~projected/kube-api-access-h8zlc:{mountpoint:/var/lib/kubelet/pods/e3cc17ea-0cec-4a71-882d-2df0457dd957/volumes/kubernetes.io~projected/kube-api-access-h8zlc major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volumes/kubernetes.io~projected/kube-api-access-phrsk:{mountpoint:/var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volumes/kubernetes.io~projected/kube-api-access-phrsk major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1cb922d-534b-4fe0-8930-b115b820d656/volumes/kubernetes.io~projected/kube-api-access-gljzz:{mountpoint:/var/lib/kubelet/pods/f1cb922d-534b-4fe0-8930-b115b820d656/volumes/kubernetes.io~projected/kube-api-access-gljzz major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2/volumes/kubernetes.io~projected/kube-api-access-r272g:{mountpoint:/var/lib/kubelet/pods/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2/volumes/kubernetes.io~projected/kube-api-access-r272g major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fa764bd7-2feb-4a11-880e-70118d1ac639/volumes/kubernetes.io~projected/kube-api-access-8grgp:{mountpoint:/var/lib/kubelet/pods/fa764bd7-2feb-4a11-880e-70118d1ac639/volumes/kubernetes.io~projected/kube-api-access-8grgp major:0 minor:232 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/54f61b1c4a248db1d2d59f8b583d8955eecc7e2966a55af3e490157456000c6c/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/cdde8f7a58d152f17a41f6ebe4618fcd97e893c9a8674c24b39003398e1f2ef0/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/da0d3cd69b8950cf29cbd02ea35ea0d40397b1cc2243fd375f025e73814aee6d/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/4de6e5a9ae792925276cd5d68117d1b2eb8e8d68feef0c810691bcea20fc714b/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/ac146bbc763ab6e93e9bf11abea6709c7bd8ced15d2bec794fbb63d1d27b1cc4/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/1888df0afaee447b3ba5953e03386078827acbab15fcfc3a417948d8d9d7e621/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/dbd69a693aee4000e913b718d164973c71e0f3c8d01f8ff102cac303fe484d0f/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/41e4050fda4f1e42ec4d5de3722f8d3bd83d68ff352d4865ecd94a12290f0eb4/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/20dc26d1425b8caeb213c73462bd10f876192bb2c66c09e612f6fcd679e6bd28/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/7e1be378453c59bc44c97f78cf19c4bf213cd477b1bcb2488546fa159d4a3a9e/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/92d0f5b037ec951540432c6b7954903fc805fbecccf8af45709a7c8f6d3a335d/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/17cfc468a2b59d5e5f0d9e9ea886cf90e52f8e9609d0f07370bff77520a8ab54/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/ebf7364c1b56c48a70396d10ed2ae3f3533c51e29ac0e062d4187ad98289bffc/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/d1f8121b16f922e2001567bc173c88274bb033788d7d7932b14ec02eddb51060/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/f1492c6ae25c492f4e2516746fab3cb1e6d4c94723f65d0ba694480345260322/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/7552aba4d5947a9b0127bc42fd2e0049d7acdaf694b5c8c9e82a5acf17fe8664/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/0adb998756415fdf8af7e31171233346d45060f35bc8eb3e0a83d846cd654d69/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/13c5e9cc81380200240eea24ae51af0d5959b7f663bb1dab8b188a08d1489892/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/2b345a3e6396922a5f6230044555c58675635dc3f0d3414afdbd429a2657c9e2/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/69ee2a815670dc6638bf134350353f6dae96463b72c4185dc6d524b00bd6f370/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/af15d508e7a19a464956a8964783198675f5abaac349111af3782462b74471bb/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/9e1fa913e52593d6027c312b1cbb5da6d4f2110c235268b1caf32dabc1e356fb/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/48af1c7ec8945fcaf643e42febb27a261186324ad9855e82c2c752ed209e86f7/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/c50189c8bba7e851299bee984a10a153643716213f20beeec243aa651168a27b/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/b0a86172230bf193598a41ed2afb841125bcb142f71083a603c266cf0ce0cd20/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/e1670c39d8994384e6754055d41bec56dd5a5d2ccef8ddcce32797367d293b6c/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/8ceb1a9efaa33035133aa7c90f5372678902907777a0c088cf6dca565e9a01ea/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/276b10842f3eb73db15bf103e30950cb4e3f6a04d51c4f3066168fcdab307490/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/562b1faeba3eac391f12176cdfef62f249edeb958b6c7bd596a0eb3f267f2463/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/edd8179768a7fbd1f89181701643be61d3a370fa1160d2a5cbbd19b84875cf95/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/7a5722b31fa2c0b42cbd9328c1e526cd39bab34f1e981b0cb25bd66c18456b59/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/a68356256d23d22ed825694235b1131ac4ec0ae7f287aa1f86b82ef859590ab5/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/bc9c8cdb0439c423e4cc91c4530fe2ed9318ecdb9401b2a459a8dfebdfa91f03/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-283:{mountpoint:/var/lib/containers/storage/overlay/6b4cf782a20a11cacc8cbe1a7ed7f4a007269391012a0862bcb6b9ced96c9024/merged major:0 minor:283 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/6f380a7ecfcf94677cdd903b53e742ae427f5c1fe1274be7cfb722d630c784c3/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/6e7df936f1d0cfdac011a284f83627c9315aa1f35b530a156ca14f0e2d96cace/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/3c93638d08cb8bfdf721267d3bf3a4d269b4e0d1fda2f8ed75440c18356fbdde/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/2d4d06f584e9b6869584753f5efe6447c6a31de80f12bda7476f97e918751902/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/3348b68d2dd4cdd46ae57fe9b60398a2285c99d50ce924901391a130562ebac2/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/ed647e1a5a0da3938e408189b92b124b6b13d1aedcba68c345e348bbf385e6ae/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/8b9c852cce26cc9465778622bca5055f0ae4023edc03a270a64df5c3696bee7f/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/ac70f2e4c1705d140816ac76d55534171a1b40180f393b87bda57f7d0f3fff7f/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/50b1ca3149079c8040cf791f05ed8ab958b01050b08ab990e896fbd7ffe9df94/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/1391678420d590221c84cae1361848f720c4cf678e5eb5cb4de088f48e760c55/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/a91a09f041eeb939be65a8a6032a3d954bfb6b2bcdca57473327509fad121501/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/586a03da2765692aa4b4bac67adbce1ff5e317eab06cc13daa0ba84dcf8d9a94/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/18d7909af82065aadf5fe38ed374ec4b427eb341e5cb992d4acc49a0e1f167b7/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/aab8da251472417f1f66aa570a21c3a47983ecc91464be522bedcad47a9672b1/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/2a96be5f85b4f5770e98f1f697b67e57b2bcbbae5807663ace2a397f2fc06e95/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/8fea606cb152628097f5d098b8832a8e85d842e526ee9c6ba85138a2dbef8daf/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-87:{mountpoint:/var/lib/containers/storage/overlay/04d188db41ede358a53ca176133637a277a5e018806b2c1d628ea013e6c8481f/merged major:0 minor:87 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/70806dbf447b5bc09de26d3566a4c7ea4cf82b6c29d939f28122b13c3053bd46/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/49ed62ae131adabe36775897c19a6b498d7a28a0953b7b632001f2b08d71113a/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/fd3c5cd81f99bef98286536718a6c73dc0a0a7fbcd8d3ab8a6cb0662e69e611e/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.697481    4849 manager.go:217] Machine: {Timestamp:2026-10-30 10:45:02.697009546 +0000 UTC m=+0.077458975 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101285 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec285afc3436fc8d0373457373205765 SystemUUID:ec285afc-3436-fc8d-0373-457373205765 BootID:e48a2206-ad52-4a0c-8fce-d4ea2536e587 Filesystems:[{Device:/var/lib/kubelet/pods/04a41ddb-fcb6-49bf-8468-deccd08a460c/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fa764bd7-2feb-4a11-880e-70118d1ac639/volumes/kubernetes.io~projected/kube-api-access-8grgp DeviceMajor:0 DeviceMinor:232 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-283 DeviceMajor:0 DeviceMinor:283 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3893e0286e812cd52687e6ac8f6a019efa64c80badf964f45f0c9b6a304c9fbd/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74498fd8124d764c596898c75bc2ba32a62d0762c83634f5b5295bcdcfa6b10e/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/418cb483-75bb-49b0-9c67-f3b70c26c2b4/volumes/kubernetes.io~projected/kube-api-access-hq49n DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~projected/kube-api-access-4hct8 DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~projected/kube-api-access-6jnkk DeviceMajor:0 DeviceMinor:231 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9e199587-19b7-4a4b-bdd0-761b7a6e475f/volumes/kubernetes.io~projected/kube-api-access-cqwn6 DeviceMajor:0 DeviceMinor:214 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ded181d7-f3c4-4324-946f-2770e8e3d0fd/volumes/kubernetes.io~projected/kube-api-access-758g5 DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/984f643d-51d1-4142-b244-f8f0980a055f/volumes/kubernetes.io~projected/kube-api-access-hxbzw DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7333057e-e1db-4970-bf86-eede9e94c06a/volumes/kubernetes.io~projected/kube-api-access-6ncc7 DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/volumes/kubernetes.io~projected/kube-api-access-tvnqf DeviceMajor:0 DeviceMinor:216 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:227 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/685fca13f89345449294586b150e424b00aab4055adb373a7e75d1cde410e851/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7333057e-e1db-4970-bf86-eede9e94c06a/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1cbcd9b8-ce20-494c-9a66-475272f2a561/volumes/kubernetes.io~projected/kube-api-access-hlkjh DeviceMajor:0 DeviceMinor:215 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:229 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~projected/kube-api-access-54d74 DeviceMajor:0 DeviceMinor:218 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a4f499333b507c41f0d264032cdea35113c6b5032c791e83932ba287418a6ffe/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2/volumes/kubernetes.io~projected/kube-api-access-r272g DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b8b2a817-e3d9-4361-98cd-c0a40c93b84d/volumes/kubernetes.io~projected/kube-api-access-z2hnd DeviceMajor:0 DeviceMinor:228 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d9916168be19512c0b0283a7a241769e5767ec2a932035458d16749cb8db9656/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~projected/kube-api-access-r2fpx DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dc78343b-907d-4222-b27d-7705a2a0f19e/volumes/kubernetes.io~projected/kube-api-access-kml79 DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a166dcbe1caca639bf7ee98a7a9d253f88e1d58d3204bd32d6e5b0aea39db67c/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/07ca9bba-a38c-48af-9a31-f71bad8bb876/volumes/kubernetes.io~projected/kube-api-access-lpl2q DeviceMajor:0 DeviceMinor:318 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d7436000ac04ff26222ea5a272d6bf8de3d44865f7d1705a41aebffd964ae6ee/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d88e5148-03db-43f8-a46f-107d8d42af0f/volumes/kubernetes.io~projected/kube-api-access-vq8tw DeviceMajor:0 DeviceMinor:243 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1906564b-67cc-491d-b1f7-2c014046a18d/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e3cc17ea-0cec-4a71-882d-2df0457dd957/volumes/kubernetes.io~projected/kube-api-access-h8zlc DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/167b7d19-70a6-43c4-a2c2-4386a574d3c3/volumes/kubernetes.io~projected/kube-api-access-b42qs DeviceMajor:0 DeviceMinor:295 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2fba2151-22de-4a03-afcb-1d89b9c32a2b/volumes/kubernetes.io~projected/kube-api-access-qshr8 DeviceMajor:0 DeviceMinor:242 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7add3842-20f6-49eb-89ad-02078f7767ef/volumes/kubernetes.io~projected/kube-api-access-vtrmh DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volumes/kubernetes.io~projected/kube-api-access-phrsk DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/91f94347ed9c3b23754ab3ea271fe0549d3f60a3ab0df7f4d1427868b99627db/userdata/shm DeviceMajor:0 DeviceMinor:222 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9e199587-19b7-4a4b-bdd0-761b7a6e475f/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:226 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a179c33d-aecd-4b72-b907-37f82a9aec15/volumes/kubernetes.io~projected/kube-api-access-qn5vl DeviceMajor:0 DeviceMinor:230 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2a552d464a9d49f9ac1dd857d287882eb8c4f367f58da3022c858f796ab898f9/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1114f0eb-6317-4004-88fa-55f60fc991a5/volumes/kubernetes.io~projected/kube-api-access-twjpp DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/997deae2ef22fb6b0f9c791a7454c5aa9821388b03fd76c40b717f3f95c4df0c/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef9d119ab837d6ffad02fb6eb0c44f4893af7ded3f78bfbb2903d34d72c6032c/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0486f6c8dac9a33be956b33582bcd1aceb4e726edee6f89f4d452115f4e3126e/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/78c49b901c1a17ecb9be8a9af3c2b860357d0cd0e2b27b036f9b41b03a23cf28/userdata/shm DeviceMajor:0 DeviceMinor:212 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/28555fb462459f87e6b3be52779d112f84230173a0f43e6b692179bc23a35d6d/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/af02cc1967f17431e480c5a28b0f6de45431f63d9e431bba6c130b0956e3d0f9/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/56a24866a1d635ce8c8a021cc89c4208c214acf6ec65ba4d59ac8ca3051f4a7a/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9bd59176-1f2c-4169-a6bc-45ca82e270eb/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-87 DeviceMajor:0 DeviceMinor:87 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d88e5148-03db-43f8-a46f-107d8d42af0f/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:241 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f1cb922d-534b-4fe0-8930-b115b820d656/volumes/kubernetes.io~projected/kube-api-access-gljzz DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0a799cd7367e63ae9838196bf3ce1143b45d0cf9f88c5f14f27e6a79ac305a23/userdata/shm DeviceMajor:0 DeviceMinor:293 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/eb883d09-b098-410d-bc1d-f68e24021d17/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb0076369a88562ee31eb074acf596cb34012dff875fdf385b0709be67695468/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b4b5be8fc34555c4ed21f6c826b523b123497f230152cedc0c172a407058e849/userdata/shm DeviceMajor:0 DeviceMinor:298 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0df37afa-77d5-4bf7-bc83-0d7636453bda/volumes/kubernetes.io~projected/kube-api-access-qlsp5 DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/04a41ddb-fcb6-49bf-8468-deccd08a460c/volumes/kubernetes.io~projected/kube-api-access-8v75s DeviceMajor:0 DeviceMinor:219 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff0e1831cc43c694d6d09a2b2fbe9a14163550eee2d9fa8c5875c489c6b44861/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0a799cd7367e63a MacAddress:be:a6:95:b0:74:95 Speed:10000 Mtu:8901} {Name:28555fb462459f8 MacAddress:2a:39:dc:0b:9a:2f Speed:10000 Mtu:8901} {Name:2a552d464a9d49f MacAddress:ba:eb:03:2d:8b:07 Speed:10000 Mtu:8901} {Name:78c49b901c1a17e MacAddress:ba:bd:59:f0:22:35 Speed:10000 Mtu:8901} {Name:91f94347ed9c3b2 MacAddress:5a:df:b8:03:17:c1 Speed:10000 Mtu:8901} {Name:997deae2ef22fb6 MacAddress:82:95:48:8f:af:0d Speed:10000 Mtu:8901} {Name:a166dcbe1caca63 MacAddress:ea:d8:b6:96:93:15 Speed:10000 Mtu:8901} {Name:af02cc1967f1743 MacAddress:62:15:e2:22:e4:a5 Speed:10000 Mtu:8901} {Name:b4b5be8fc34555c MacAddress:fe:19:cf:13:79:00 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c0:c6:5a:91 Speed:0 Mtu:9001} {Name:br-int MacAddress:ae:99:9e:6d:9b:d9 Speed:0 Mtu:8901} {Name:d9916168be19512 MacAddress:2a:2f:c6:1e:6b:f9 Speed:10000 Mtu:8901} {Name:eb0076369a88562 MacAddress:ce:1d:d7:3b:0a:17 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c0:c6:5a:91 Speed:0 Mtu:9001} {Name:ff0e1831cc43c69 MacAddress:12:aa:29:80:5c:b5 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:66:3c:b8:df:e5:e7 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:d2:d5:e8:6b:fc:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.697822    4849 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.697862    4849 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698190    4849 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698218    4849 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698329    4849 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698336    4849 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698347    4849 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698366    4849 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 10:45:02.698564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698385    4849 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 10:45:02.698976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698676    4849 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 10:45:02.698976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698773    4849 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 10:45:02.698976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698805    4849 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 10:45:02.698976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698816    4849 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 10:45:02.698976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698832    4849 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 10:45:02.698976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698840    4849 kubelet.go:405] "Adding apiserver pod source"
Oct 30 10:45:02.698976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.698849    4849 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 10:45:02.699710 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.699696    4849 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 10:45:02.699710 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.699709    4849 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 10:45:02.700188 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.700152    4849 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 10:45:02.702716 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.702703    4849 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 10:45:02.702716 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.702716    4849 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 10:45:02.702835 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.702826    4849 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 10:45:02.703004 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.702990    4849 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 10:45:02.703053 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703038    4849 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 10:45:02.703155 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703145    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 10:45:02.703185 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703165    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 10:45:02.703185 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703173    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 10:45:02.703234 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703193    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 10:45:02.703234 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703199    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 10:45:02.703234 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703205    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 10:45:02.703234 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703211    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 10:45:02.703234 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703217    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 10:45:02.703234 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703224    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 10:45:02.703234 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703230    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 10:45:02.703412 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703243    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 10:45:02.703412 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703252    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 10:45:02.703412 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703297    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 10:45:02.703412 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.703303    4849 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 10:45:02.706660 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.706647    4849 server.go:1263] "Started kubelet"
Oct 30 10:45:02.706773 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.706744    4849 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 10:45:02.707408 ip-10-0-140-230 systemd[1]: Started Kubernetes Kubelet.
Oct 30 10:45:02.711051 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.708989    4849 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 10:45:02.711051 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.709133    4849 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 10:45:02.711051 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.709538    4849 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 10:45:02.711051 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.709645    4849 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 10:45:02.713562 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.713543    4849 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 10:45:02.715062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.713565    4849 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 10:45:02.715062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.713663    4849 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 10:45:02.715062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.713674    4849 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 10:45:02.715062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.713752    4849 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 10:39:24 +0000 UTC" deadline="2028-04-01 19:18:05.012346205 +0000 UTC"
Oct 30 10:45:02.715062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.713773    4849 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12464h33m2.29857649s"
Oct 30 10:45:02.715062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.713776    4849 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 10:45:02.715062 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:02.714189    4849 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-230.ec2.internal\" not found"
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716307    4849 factory.go:55] Registering systemd factory
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716332    4849 factory.go:223] Registration of the systemd container factory successfully
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716365    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04a41ddb-fcb6-49bf-8468-deccd08a460c" volumeName="kubernetes.io/secret/04a41ddb-fcb6-49bf-8468-deccd08a460c-serving-cert" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716407    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fba2151-22de-4a03-afcb-1d89b9c32a2b" volumeName="kubernetes.io/configmap/2fba2151-22de-4a03-afcb-1d89b9c32a2b-config-volume" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716421    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" volumeName="kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716433    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" volumeName="kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-config" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716445    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bd59176-1f2c-4169-a6bc-45ca82e270eb" volumeName="kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-default-certificate" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716457    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04a41ddb-fcb6-49bf-8468-deccd08a460c" volumeName="kubernetes.io/projected/04a41ddb-fcb6-49bf-8468-deccd08a460c-kube-api-access-8v75s" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716467    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1114f0eb-6317-4004-88fa-55f60fc991a5" volumeName="kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-daemon-config" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716479    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657" volumeName="kubernetes.io/secret/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-agent-certs" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716495    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="984f643d-51d1-4142-b244-f8f0980a055f" volumeName="kubernetes.io/projected/984f643d-51d1-4142-b244-f8f0980a055f-kube-api-access-hxbzw" seLinuxMountContext=""
Oct 30 10:45:02.716498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716507    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d88e5148-03db-43f8-a46f-107d8d42af0f" volumeName="kubernetes.io/projected/d88e5148-03db-43f8-a46f-107d8d42af0f-kube-api-access-vq8tw" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716519    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc78343b-907d-4222-b27d-7705a2a0f19e" volumeName="kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716531    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1906564b-67cc-491d-b1f7-2c014046a18d" volumeName="kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-etc-tuned" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716543    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7333057e-e1db-4970-bf86-eede9e94c06a" volumeName="kubernetes.io/configmap/7333057e-e1db-4970-bf86-eede9e94c06a-config" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716555    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7333057e-e1db-4970-bf86-eede9e94c06a" volumeName="kubernetes.io/projected/7333057e-e1db-4970-bf86-eede9e94c06a-kube-api-access-6ncc7" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716589    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7add3842-20f6-49eb-89ad-02078f7767ef" volumeName="kubernetes.io/configmap/7add3842-20f6-49eb-89ad-02078f7767ef-serviceca" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716611    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bd59176-1f2c-4169-a6bc-45ca82e270eb" volumeName="kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-stats-auth" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716633    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e199587-19b7-4a4b-bdd0-761b7a6e475f" volumeName="kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-service-ca-bundle" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716578    4849 factory.go:153] Registering CRI-O factory
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716661    4849 factory.go:223] Registration of the crio container factory successfully
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716731    4849 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716770    4849 factory.go:103] Registering Raw factory
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716647    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e199587-19b7-4a4b-bdd0-761b7a6e475f" volumeName="kubernetes.io/projected/9e199587-19b7-4a4b-bdd0-761b7a6e475f-kube-api-access-cqwn6" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716788    4849 manager.go:1199] Started watching for new ooms in manager
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716794    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc612ba8-81c6-4d10-aecf-f8efdc6fea4f" volumeName="kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716807    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9fb55617-3df2-401d-876b-2fc95f902c53" volumeName="kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716828    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a179c33d-aecd-4b72-b907-37f82a9aec15" volumeName="kubernetes.io/configmap/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-config" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716839    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1cbcd9b8-ce20-494c-9a66-475272f2a561" volumeName="kubernetes.io/configmap/1cbcd9b8-ce20-494c-9a66-475272f2a561-telemetry-config" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716849    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" volumeName="kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716873    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e3cc17ea-0cec-4a71-882d-2df0457dd957" volumeName="kubernetes.io/projected/e3cc17ea-0cec-4a71-882d-2df0457dd957-kube-api-access-h8zlc" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716885    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9fb55617-3df2-401d-876b-2fc95f902c53" volumeName="kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716898    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1114f0eb-6317-4004-88fa-55f60fc991a5" volumeName="kubernetes.io/projected/1114f0eb-6317-4004-88fa-55f60fc991a5-kube-api-access-twjpp" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716923    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fba2151-22de-4a03-afcb-1d89b9c32a2b" volumeName="kubernetes.io/empty-dir/2fba2151-22de-4a03-afcb-1d89b9c32a2b-tmp-dir" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716940    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7add3842-20f6-49eb-89ad-02078f7767ef" volumeName="kubernetes.io/projected/7add3842-20f6-49eb-89ad-02078f7767ef-kube-api-access-vtrmh" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716962    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657" volumeName="kubernetes.io/configmap/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-konnectivity-ca" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716979    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9fb55617-3df2-401d-876b-2fc95f902c53" volumeName="kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.716994    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8b2a817-e3d9-4361-98cd-c0a40c93b84d" volumeName="kubernetes.io/projected/b8b2a817-e3d9-4361-98cd-c0a40c93b84d-kube-api-access-z2hnd" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717010    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb883d09-b098-410d-bc1d-f68e24021d17" volumeName="kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-config" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717043    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9fb55617-3df2-401d-876b-2fc95f902c53" volumeName="kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717059    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc78343b-907d-4222-b27d-7705a2a0f19e" volumeName="kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717071    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb883d09-b098-410d-bc1d-f68e24021d17" volumeName="kubernetes.io/projected/eb883d09-b098-410d-bc1d-f68e24021d17-kube-api-access-phrsk" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717087    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb883d09-b098-410d-bc1d-f68e24021d17" volumeName="kubernetes.io/secret/eb883d09-b098-410d-bc1d-f68e24021d17-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717098    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f384be3a-0f13-4b0a-ba1b-d63d2095b9d2" volumeName="kubernetes.io/projected/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2-kube-api-access-r272g" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717107    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="418cb483-75bb-49b0-9c67-f3b70c26c2b4" volumeName="kubernetes.io/projected/418cb483-75bb-49b0-9c67-f3b70c26c2b4-kube-api-access-hq49n" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717119    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" volumeName="kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717128    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9fb55617-3df2-401d-876b-2fc95f902c53" volumeName="kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717138    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a179c33d-aecd-4b72-b907-37f82a9aec15" volumeName="kubernetes.io/projected/a179c33d-aecd-4b72-b907-37f82a9aec15-kube-api-access-qn5vl" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717148    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1cb922d-534b-4fe0-8930-b115b820d656" volumeName="kubernetes.io/configmap/f1cb922d-534b-4fe0-8930-b115b820d656-iptables-alerter-script" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717158    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07ca9bba-a38c-48af-9a31-f71bad8bb876" volumeName="kubernetes.io/empty-dir/07ca9bba-a38c-48af-9a31-f71bad8bb876-data-volume" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717166    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" volumeName="kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-trusted-ca" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717175    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" volumeName="kubernetes.io/secret/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-serving-cert" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717186    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e199587-19b7-4a4b-bdd0-761b7a6e475f" volumeName="kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717196    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9fb55617-3df2-401d-876b-2fc95f902c53" volumeName="kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717208    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc78343b-907d-4222-b27d-7705a2a0f19e" volumeName="kubernetes.io/projected/dc78343b-907d-4222-b27d-7705a2a0f19e-kube-api-access-kml79" seLinuxMountContext=""
Oct 30 10:45:02.717428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717211    4849 manager.go:319] Starting recovery of all containers
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.717215    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fa764bd7-2feb-4a11-880e-70118d1ac639" volumeName="kubernetes.io/projected/fa764bd7-2feb-4a11-880e-70118d1ac639-kube-api-access-8grgp" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719069    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" volumeName="kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719129    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07ca9bba-a38c-48af-9a31-f71bad8bb876" volumeName="kubernetes.io/projected/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-api-access-lpl2q" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719145    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded181d7-f3c4-4324-946f-2770e8e3d0fd" volumeName="kubernetes.io/projected/ded181d7-f3c4-4324-946f-2770e8e3d0fd-kube-api-access-758g5" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719159    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="167b7d19-70a6-43c4-a2c2-4386a574d3c3" volumeName="kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719172    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" volumeName="kubernetes.io/projected/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-kube-api-access-tvnqf" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719184    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0df37afa-77d5-4bf7-bc83-0d7636453bda" volumeName="kubernetes.io/projected/0df37afa-77d5-4bf7-bc83-0d7636453bda-kube-api-access-qlsp5" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719196    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1906564b-67cc-491d-b1f7-2c014046a18d" volumeName="kubernetes.io/projected/1906564b-67cc-491d-b1f7-2c014046a18d-kube-api-access-r2fpx" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719218    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1cbcd9b8-ce20-494c-9a66-475272f2a561" volumeName="kubernetes.io/projected/1cbcd9b8-ce20-494c-9a66-475272f2a561-kube-api-access-hlkjh" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719229    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fba2151-22de-4a03-afcb-1d89b9c32a2b" volumeName="kubernetes.io/projected/2fba2151-22de-4a03-afcb-1d89b9c32a2b-kube-api-access-qshr8" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719242    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" volumeName="kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719254    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e199587-19b7-4a4b-bdd0-761b7a6e475f" volumeName="kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-tmp" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719267    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc78343b-907d-4222-b27d-7705a2a0f19e" volumeName="kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-binary-copy" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719298    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7333057e-e1db-4970-bf86-eede9e94c06a" volumeName="kubernetes.io/secret/7333057e-e1db-4970-bf86-eede9e94c06a-serving-cert" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719310    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bd59176-1f2c-4169-a6bc-45ca82e270eb" volumeName="kubernetes.io/projected/9bd59176-1f2c-4169-a6bc-45ca82e270eb-kube-api-access-54d74" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719321    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e199587-19b7-4a4b-bdd0-761b7a6e475f" volumeName="kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-snapshots" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719332    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d88e5148-03db-43f8-a46f-107d8d42af0f" volumeName="kubernetes.io/secret/d88e5148-03db-43f8-a46f-107d8d42af0f-cert" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719343    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ded181d7-f3c4-4324-946f-2770e8e3d0fd" volumeName="kubernetes.io/empty-dir/ded181d7-f3c4-4324-946f-2770e8e3d0fd-tmp-dir" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719354    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb883d09-b098-410d-bc1d-f68e24021d17" volumeName="kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-env-overrides" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719365    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb883d09-b098-410d-bc1d-f68e24021d17" volumeName="kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719378    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04a41ddb-fcb6-49bf-8468-deccd08a460c" volumeName="kubernetes.io/configmap/04a41ddb-fcb6-49bf-8468-deccd08a460c-config" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719388    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3138c05b-7ff7-463e-9d91-ea7fab54a541" volumeName="kubernetes.io/configmap/3138c05b-7ff7-463e-9d91-ea7fab54a541-nginx-conf" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719412    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9fb55617-3df2-401d-876b-2fc95f902c53" volumeName="kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719428    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1114f0eb-6317-4004-88fa-55f60fc991a5" volumeName="kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-cni-binary-copy" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719439    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" volumeName="kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719450    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1906564b-67cc-491d-b1f7-2c014046a18d" volumeName="kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-tmp" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719461    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" volumeName="kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719472    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e199587-19b7-4a4b-bdd0-761b7a6e475f" volumeName="kubernetes.io/secret/9e199587-19b7-4a4b-bdd0-761b7a6e475f-serving-cert" seLinuxMountContext=""
Oct 30 10:45:02.719477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719499    4849 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1cb922d-534b-4fe0-8930-b115b820d656" volumeName="kubernetes.io/projected/f1cb922d-534b-4fe0-8930-b115b820d656-kube-api-access-gljzz" seLinuxMountContext=""
Oct 30 10:45:02.720795 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719511    4849 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 10:45:02.720795 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.719518    4849 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 10:45:02.730131 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.730112    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/1.log"
Oct 30 10:45:02.730236 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.730150    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/1.log"
Oct 30 10:45:02.745650 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.745627    4849 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 10:45:02.746827 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.746804    4849 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 10:45:02.746909 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.746834    4849 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 10:45:02.746909 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.746857    4849 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 10:45:02.746909 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:02.746901    4849 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 10:45:02.766202 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.766181    4849 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 10:45:02.766952 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.766927    4849 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:45:02.767061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.766951    4849 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:45:02.770131 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.770102    4849 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="355a1810a1f865b52692daf925904f3e88341445dca434dea6ef25755b2e648e" exitCode=0
Oct 30 10:45:02.770131 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.770131    4849 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="3c7bbed30f3252bf5a62a48c879e26b104cd0ac270610af06a05628d9dbe0ddf" exitCode=0
Oct 30 10:45:02.770281 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.770141    4849 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="39310ff80c1f589aa10599656e8fe7f452cfbc2039c82572ea0928ae8f715340" exitCode=0
Oct 30 10:45:02.770281 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.770152    4849 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="23869dd34f996ae015bf48d600bab87cc87fc48bd8b9ae57156690356392a037" exitCode=0
Oct 30 10:45:02.770281 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.770163    4849 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="7d82ffd81a0b526c5a74fd866902288286430c598e26bb3d32cd2ead09aa69d5" exitCode=0
Oct 30 10:45:02.770281 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.770171    4849 generic.go:356] "Generic (PLEG): container finished" podID="dc78343b-907d-4222-b27d-7705a2a0f19e" containerID="c74f5d83716017d6f53c6bc12343be55fc06c240df93484ea630662362cf6e55" exitCode=0
Oct 30 10:45:02.781129 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.781108    4849 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:45:02.782752 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.782732    4849 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 10:45:02.793582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.793559    4849 generic.go:356] "Generic (PLEG): container finished" podID="356d8c2a38ca6b7521952074b996a2cc" containerID="ef55900939de29facc26b1680919df7ebadd34de496d33d38281ffe8e127bc4d" exitCode=0
Oct 30 10:45:02.795096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.795081    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/1.log"
Oct 30 10:45:02.795195 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.795102    4849 generic.go:356] "Generic (PLEG): container finished" podID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" containerID="f8912bf9c29e99061121aba6a3b280560ad379cf25a59577044eb706d3d8e5f9" exitCode=255
Oct 30 10:45:02.804895 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.804854    4849 manager.go:324] Recovery completed
Oct 30 10:45:02.824334 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824310    4849 cpu_manager.go:226] "Starting" policy="none"
Oct 30 10:45:02.824334 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824328    4849 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 10:45:02.824426 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824346    4849 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 10:45:02.824464 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824451    4849 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 10:45:02.824504 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824465    4849 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 10:45:02.824504 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824480    4849 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 10:45:02.824504 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824485    4849 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 10:45:02.824504 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824490    4849 policy_none.go:50] "Start"
Oct 30 10:45:02.824504 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824498    4849 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 10:45:02.824631 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824507    4849 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 10:45:02.824631 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824623    4849 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 10:45:02.824631 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824630    4849 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 10:45:02.824709 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.824635    4849 policy_none.go:44] "Start"
Oct 30 10:45:02.828777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.828753    4849 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 10:45:02.832168 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.832155    4849 manager.go:341] "Starting Device Plugin manager"
Oct 30 10:45:02.832594 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:02.832182    4849 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 10:45:02.832663 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.832598    4849 server.go:87] "Starting device plugin registration server"
Oct 30 10:45:02.832868 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.832855    4849 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 10:45:02.832928 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.832870    4849 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 10:45:02.832989 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.832954    4849 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 10:45:02.833101 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.833071    4849 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 10:45:02.833101 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.833104    4849 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 10:45:02.833248 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.833111    4849 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 10:45:02.833248 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.833157    4849 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T10:45:02.833095202Z","UUID":"91fe514d-848d-4c06-828b-c039087356e5","Handler":null,"Name":"","Endpoint":""}
Oct 30 10:45:02.834436 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.834417    4849 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 10:45:02.834541 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.834442    4849 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 10:45:02.834541 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:02.834440    4849 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 10:45:02.847034 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.846989    4849 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"]
Oct 30 10:45:02.847584 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.847569    4849 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.847649 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.847587    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" event={"ID":"d42823a77ed791ee517181c1c853da97","Type":"ContainerStarted","Data":"38d67bd062a057c0b7ebcd0ab12ed4bee5fff785966d50fdf5f0e5bac722036d"}
Oct 30 10:45:02.847649 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.847625    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal" event={"ID":"d42823a77ed791ee517181c1c853da97","Type":"ContainerStarted","Data":"f690dcc087e11643533425375b5fccfd5301fa8ca8cfce7549a090aecd94b771"}
Oct 30 10:45:02.847751 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.847654    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"356d8c2a38ca6b7521952074b996a2cc","Type":"ContainerStarted","Data":"6d9f3be0b99abc8668e73a5030faf4aaf0443221aa9cab3695b65469f067df82"}
Oct 30 10:45:02.847751 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.847665    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"356d8c2a38ca6b7521952074b996a2cc","Type":"ContainerDied","Data":"ef55900939de29facc26b1680919df7ebadd34de496d33d38281ffe8e127bc4d"}
Oct 30 10:45:02.847751 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.847681    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal" event={"ID":"356d8c2a38ca6b7521952074b996a2cc","Type":"ContainerStarted","Data":"054c10d4167efa5ded051e282a0d8bb2b066a31945e6d5c9f9a7658368fcd31c"}
Oct 30 10:45:02.847844 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.847794    4849 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.873317 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.873296    4849 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 10:45:02.873406 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:02.873346    4849 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.873846 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.873833    4849 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 10:45:02.873878 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:02.873864    4849 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.933513 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.933491    4849 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 10:45:02.934699 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.934683    4849 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 10:45:02.934777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.934712    4849 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 10:45:02.934777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.934726    4849 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-230.ec2.internal" event="NodeHasSufficientPID"
Oct 30 10:45:02.934777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.934774    4849 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.967051 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.967014    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.967137 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.967071    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d42823a77ed791ee517181c1c853da97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"d42823a77ed791ee517181c1c853da97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.967137 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.967098    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.972277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.972259    4849 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-230.ec2.internal"
Oct 30 10:45:02.972364 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:02.972311    4849 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.068037 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.067999    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.068137 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.068061    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.068137 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.068082    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.068137 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.068097    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d42823a77ed791ee517181c1c853da97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"d42823a77ed791ee517181c1c853da97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.068235 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.068137    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d42823a77ed791ee517181c1c853da97-config\") pod \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" (UID: \"d42823a77ed791ee517181c1c853da97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.068235 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.068141    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/356d8c2a38ca6b7521952074b996a2cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" (UID: \"356d8c2a38ca6b7521952074b996a2cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.699640 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.699610    4849 apiserver.go:52] "Watching apiserver"
Oct 30 10:45:03.799399 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.799380    4849 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.799550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.799536    4849 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.852067 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.852042    4849 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 10:45:03.852144 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.852065    4849 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 10:45:03.852144 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:03.852086    4849 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.852144 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:03.852109    4849 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-230.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal"
Oct 30 10:45:03.852318 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.852301    4849 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 10:45:03.852876 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.852855    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lw745","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-230.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-5sl2r","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6","openshift-network-operator/iptables-alerter-2kxdq","kube-system/kube-apiserver-proxy-ip-10-0-140-230.ec2.internal","openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst","openshift-console-operator/console-operator-7cd56b8dbc-2hmr5","openshift-ingress-canary/ingress-canary-7kq9h","openshift-ingress/router-default-65668b76cf-ckhgx","openshift-insights/insights-operator-7c4986d449-v69fn","kube-system/kas-connection-checker-685895c7c9-xfp2x","openshift-multus/multus-2c8hg","kube-system/global-pull-secret-syncer-lkmz6","openshift-image-registry/node-ca-qxz95","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz","openshift-network-console/networking-console-plugin-86c478bb88-fdv8p","openshift-dns/dns-default-mx9gc","openshift-network-diagnostics/network-check-target-ccnv5","openshift-multus/multus-additional-cni-plugins-4k458","kube-system/kas-connection-checker-685895c7c9-8pgqv","openshift-image-registry/image-registry-794b85596b-gvxrs","openshift-image-registry/image-registry-67bdbf894c-qszt7","kube-system/kas-connection-checker-685895c7c9-r9ns7","openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz","kube-system/konnectivity-agent-q6ddt","openshift-multus/network-metrics-daemon-hr78s","openshift-dns/node-resolver-887wc","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt","openshift-cluster-node-tuning-operator/tuned-rnv69","openshift-insights/insights-runtime-extractor-2kxlm"]
Oct 30 10:45:03.854323 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.854298    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:03.854890 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.854871    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.856117 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.856095    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:03.856947 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.856929    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.857424 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.857408    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:03.858695 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.858681    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:03.867147 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.867121    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.867282 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.867253    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:03.868792 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.868774    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:03.870270 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.870254    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.870404 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.870389    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.871081 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.871069    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.871133 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.871119    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 10:45:03.871605 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.871592    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 10:45:03.872737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.872714    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 10:45:03.872737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.872729    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 10:45:03.884336 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.884315    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 10:45:03.884336 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.884330    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 10:45:03.884455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.884383    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 10:45:03.885638 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885624    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 10:45:03.885638 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885631    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 10:45:03.885709 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885635    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 10:45:03.885738 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885727    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.885782 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885772    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.885818 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885774    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.885853 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885830    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.885882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.885866    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.886115 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886095    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 10:45:03.886115 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886097    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 10:45:03.886240 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886177    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 10:45:03.886294 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886237    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.886415 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886401    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.886461 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886422    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 10:45:03.886461 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886449    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 10:45:03.886530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886497    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sssm5\""
Oct 30 10:45:03.886657 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886637    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 10:45:03.886901 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886879    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 10:45:03.886901 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.886883    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 10:45:03.887373 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887356    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 10:45:03.887447 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887422    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 10:45:03.887527 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887512    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s5t5b\""
Oct 30 10:45:03.887588 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887523    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.887588 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887530    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 10:45:03.887588 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887559    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 10:45:03.887833 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887626    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 10:45:03.887833 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887723    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 10:45:03.887833 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887730    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 10:45:03.887833 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887725    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 10:45:03.888061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887835    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.888061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887891    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.888061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887898    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 10:45:03.888061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887938    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c9pg5\""
Oct 30 10:45:03.888061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.887997    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lntjm\""
Oct 30 10:45:03.888296 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888077    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.888296 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888259    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 10:45:03.888377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888360    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-vmnnx\""
Oct 30 10:45:03.888435 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888414    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 10:45:03.888492 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888437    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 10:45:03.888492 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888439    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-mhmcf\""
Oct 30 10:45:03.888492 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888459    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 10:45:03.888492 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888463    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5fdc6\""
Oct 30 10:45:03.888492 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888488    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888647    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-472bd\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888649    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888662    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xt7nj\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888672    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888658    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888705    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888715    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-55bss\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888722    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgw7c\""
Oct 30 10:45:03.888784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888729    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.888989    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889054    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889098    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889108    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889117    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9pzxh\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889125    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889099    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889135    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889144    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889141    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889176    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.889251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.889195    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.898117 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898088    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.898117 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898106    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.898481 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898462    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898468    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898486    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898492    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898502    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898469    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-8g6wd\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898514    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898487    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898518    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898519    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 10:45:03.898582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898578    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 10:45:03.898988 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898589    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-r88l6\""
Oct 30 10:45:03.898988 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898595    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gnsmw\""
Oct 30 10:45:03.898988 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898737    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.898988 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.898742    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 10:45:03.900086 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.900067    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ghh8s\""
Oct 30 10:45:03.900193 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.900092    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vwt65\""
Oct 30 10:45:03.900193 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.900128    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 10:45:03.900193 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.900136    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:45:03.900193 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.900172    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 10:45:03.900339 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.900214    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 10:45:03.918500 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.918479    4849 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 10:45:03.972920 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.972892    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-systemd-units\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.972920 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.972925    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.973052 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.972944    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.973052 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.972963    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-kubernetes\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.973052 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973034    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.973152 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973073    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-conf\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.973184 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973174    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-agent-certs\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:45:03.973222 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973202    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpl2q\" (UniqueName: \"kubernetes.io/projected/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-api-access-lpl2q\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:03.973255 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973236    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.973317 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973305    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-conf-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.973356 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973338    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:03.973403 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973358    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-snapshots\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.973446 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973408    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jnkk\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.973485 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973465    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zlc\" (UniqueName: \"kubernetes.io/projected/e3cc17ea-0cec-4a71-882d-2df0457dd957-kube-api-access-h8zlc\") pod \"kas-connection-checker-685895c7c9-r9ns7\" (UID: \"e3cc17ea-0cec-4a71-882d-2df0457dd957\") " pod="kube-system/kas-connection-checker-685895c7c9-r9ns7"
Oct 30 10:45:03.973485 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973469    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-snapshots\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.973571 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973552    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-agent-certs\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:45:03.973609 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973575    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kml79\" (UniqueName: \"kubernetes.io/projected/dc78343b-907d-4222-b27d-7705a2a0f19e-kube-api-access-kml79\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.973724 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973705    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn5vl\" (UniqueName: \"kubernetes.io/projected/a179c33d-aecd-4b72-b907-37f82a9aec15-kube-api-access-qn5vl\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:03.973770 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973740    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-netns\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.973806 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973768    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.973844 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973804    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtrmh\" (UniqueName: \"kubernetes.io/projected/7add3842-20f6-49eb-89ad-02078f7767ef-kube-api-access-vtrmh\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:45:03.973844 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973828    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded181d7-f3c4-4324-946f-2770e8e3d0fd-hosts-file\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:45:03.973918 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973844    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:03.973918 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973908    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-service-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.973979 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973931    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.973979 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973949    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cb922d-534b-4fe0-8930-b115b820d656-host-slash\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:45:03.973979 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973963    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-var-lib-kubelet\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.974105 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.973994    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-multus\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.974105 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974002    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.974105 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974042    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-sys-fs\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:03.974105 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974076    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-cni-binary-copy\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.974105 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974100    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-tmp\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.974253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974123    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-service-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.974253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974126    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-multus-certs\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.974253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974149    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-etc-kubernetes\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.974253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974171    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e199587-19b7-4a4b-bdd0-761b7a6e475f-tmp\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.974253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974200    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-slash\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.974253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974214    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-var-lib-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.974253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974230    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-script-lib\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.974459 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974266    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qshr8\" (UniqueName: \"kubernetes.io/projected/2fba2151-22de-4a03-afcb-1d89b9c32a2b-kube-api-access-qshr8\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:03.974459 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974292    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-default-certificate\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.974459 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974401    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:03.974564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974460    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-script-lib\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.974564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974474    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:03.974564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974509    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-cni-binary-copy\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.974653 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974583    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-default-certificate\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.974653 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974594    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v75s\" (UniqueName: \"kubernetes.io/projected/04a41ddb-fcb6-49bf-8468-deccd08a460c-kube-api-access-8v75s\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:45:03.974653 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974627    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/ded181d7-f3c4-4324-946f-2770e8e3d0fd-kube-api-access-758g5\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:45:03.974745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974652    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cb922d-534b-4fe0-8930-b115b820d656-iptables-alerter-script\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:45:03.974745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974676    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ca9bba-a38c-48af-9a31-f71bad8bb876-data-volume\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:03.974745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974684    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-original-pull-secret\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:03.974745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974700    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-systemd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.974745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974724    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-bin\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.974906 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974749    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1cbcd9b8-ce20-494c-9a66-475272f2a561-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:03.974906 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974770    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ca9bba-a38c-48af-9a31-f71bad8bb876-data-volume\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:03.974906 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974797    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-serving-cert\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:03.974906 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974824    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-sys\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.974906 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974852    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-netns\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.974906 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974873    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-kubelet\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.974906 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974896    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-ovn\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.975146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974941    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cb922d-534b-4fe0-8930-b115b820d656-iptables-alerter-script\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:45:03.975146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974951    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-serving-cert\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:03.975146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974958    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phrsk\" (UniqueName: \"kubernetes.io/projected/eb883d09-b098-410d-bc1d-f68e24021d17-kube-api-access-phrsk\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.975146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974967    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1cbcd9b8-ce20-494c-9a66-475272f2a561-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:03.975146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.974984    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3138c05b-7ff7-463e-9d91-ea7fab54a541-nginx-conf\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:03.975146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975126    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded181d7-f3c4-4324-946f-2770e8e3d0fd-tmp-dir\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:45:03.975330 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975162    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-socket-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:03.975330 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975173    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3138c05b-7ff7-463e-9d91-ea7fab54a541-nginx-conf\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:03.975330 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975193    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.975330 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975232    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded181d7-f3c4-4324-946f-2770e8e3d0fd-tmp-dir\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:45:03.975330 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975270    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq49n\" (UniqueName: \"kubernetes.io/projected/418cb483-75bb-49b0-9c67-f3b70c26c2b4-kube-api-access-hq49n\") pod \"kas-connection-checker-685895c7c9-xfp2x\" (UID: \"418cb483-75bb-49b0-9c67-f3b70c26c2b4\") " pod="kube-system/kas-connection-checker-685895c7c9-xfp2x"
Oct 30 10:45:03.975481 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975373    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlsp5\" (UniqueName: \"kubernetes.io/projected/0df37afa-77d5-4bf7-bc83-0d7636453bda-kube-api-access-qlsp5\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:03.975481 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975402    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-node-log\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.975481 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975409    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.975481 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975468    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlkjh\" (UniqueName: \"kubernetes.io/projected/1cbcd9b8-ce20-494c-9a66-475272f2a561-kube-api-access-hlkjh\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:03.975630 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975508    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqwn6\" (UniqueName: \"kubernetes.io/projected/9e199587-19b7-4a4b-bdd0-761b7a6e475f-kube-api-access-cqwn6\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.975630 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975530    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:03.975630 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975547    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:03.975630 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975565    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.975764 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975666    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7add3842-20f6-49eb-89ad-02078f7767ef-serviceca\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:45:03.975764 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975697    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:03.975764 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975725    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.975851 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975778    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.975851 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975801    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-cnibin\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.975851 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975821    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.975851 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975779    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:03.975851 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975838    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7add3842-20f6-49eb-89ad-02078f7767ef-host\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:45:03.975995 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975872    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7add3842-20f6-49eb-89ad-02078f7767ef-serviceca\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:45:03.975995 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975904    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:03.975995 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975923    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:03.975995 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975927    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.975995 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975960    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-trusted-ca\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:03.975995 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975979    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.976198 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.975996    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-device-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:03.976198 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976053    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-netd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.976198 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976087    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.976198 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976148    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba2151-22de-4a03-afcb-1d89b9c32a2b-tmp-dir\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:03.976198 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976183    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ncc7\" (UniqueName: \"kubernetes.io/projected/7333057e-e1db-4970-bf86-eede9e94c06a-kube-api-access-6ncc7\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976211    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba2151-22de-4a03-afcb-1d89b9c32a2b-tmp-dir\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976241    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-stats-auth\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976280    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-dbus\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976309    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976318    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-trusted-ca\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976339    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976379    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-etc-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976431    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.976455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976442    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-stats-auth\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976506    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-k8s-cni-cncf-io\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976541    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976557    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r272g\" (UniqueName: \"kubernetes.io/projected/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2-kube-api-access-r272g\") pod \"kas-connection-checker-685895c7c9-8pgqv\" (UID: \"f384be3a-0f13-4b0a-ba1b-d63d2095b9d2\") " pod="kube-system/kas-connection-checker-685895c7c9-8pgqv"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976566    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976578    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7333057e-e1db-4970-bf86-eede9e94c06a-serving-cert\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976581    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976604    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976638    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2fpx\" (UniqueName: \"kubernetes.io/projected/1906564b-67cc-491d-b1f7-2c014046a18d-kube-api-access-r2fpx\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976657    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb883d09-b098-410d-bc1d-f68e24021d17-ovn-node-metrics-cert\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976696    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-socket-dir-parent\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976701    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7333057e-e1db-4970-bf86-eede9e94c06a-serving-cert\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976725    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976750    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a41ddb-fcb6-49bf-8468-deccd08a460c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976766    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e199587-19b7-4a4b-bdd0-761b7a6e475f-serving-cert\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976784    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976794    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb883d09-b098-410d-bc1d-f68e24021d17-ovn-node-metrics-cert\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976800    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-modprobe-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976816    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-lib-modules\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976865    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-system-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976889    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.976898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976901    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/04a41ddb-fcb6-49bf-8468-deccd08a460c-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976925    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-config\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976945    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-registration-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.976960    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977007    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc78343b-907d-4222-b27d-7705a2a0f19e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977006    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysconfig\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977073    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-etc-tuned\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977089    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-hostroot\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977118    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977124    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977151    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e199587-19b7-4a4b-bdd0-761b7a6e475f-serving-cert\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977158    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-ovnkube-config\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977177    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-etc-tuned\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977209    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twjpp\" (UniqueName: \"kubernetes.io/projected/1114f0eb-6317-4004-88fa-55f60fc991a5-kube-api-access-twjpp\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977254    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7333057e-e1db-4970-bf86-eede9e94c06a-config\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977322    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grgp\" (UniqueName: \"kubernetes.io/projected/fa764bd7-2feb-4a11-880e-70118d1ac639-kube-api-access-8grgp\") pod \"network-check-source-6f78d59f99-w7wqz\" (UID: \"fa764bd7-2feb-4a11-880e-70118d1ac639\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977341    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54d74\" (UniqueName: \"kubernetes.io/projected/9bd59176-1f2c-4169-a6bc-45ca82e270eb-kube-api-access-54d74\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977386    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977424    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d88e5148-03db-43f8-a46f-107d8d42af0f-cert\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977452    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-systemd\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977476    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a41ddb-fcb6-49bf-8468-deccd08a460c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977501    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-config\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977531    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7333057e-e1db-4970-bf86-eede9e94c06a-config\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977541    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq8tw\" (UniqueName: \"kubernetes.io/projected/d88e5148-03db-43f8-a46f-107d8d42af0f-kube-api-access-vq8tw\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977563    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-konnectivity-ca\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977675    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977699    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d88e5148-03db-43f8-a46f-107d8d42af0f-cert\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977713    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977717    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657-konnectivity-ca\") pod \"konnectivity-agent-q6ddt\" (UID: \"8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657\") " pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977750    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-env-overrides\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977753    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/04a41ddb-fcb6-49bf-8468-deccd08a460c-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977798    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977826    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-daemon-config\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977828    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-config\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977859    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvnqf\" (UniqueName: \"kubernetes.io/projected/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-kube-api-access-tvnqf\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977905    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb883d09-b098-410d-bc1d-f68e24021d17-env-overrides\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977911    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2hnd\" (UniqueName: \"kubernetes.io/projected/b8b2a817-e3d9-4361-98cd-c0a40c93b84d-kube-api-access-z2hnd\") pod \"volume-data-source-validator-58c9f5f46d-5xkvn\" (UID: \"b8b2a817-e3d9-4361-98cd-c0a40c93b84d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977943    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-bin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977969    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-log-socket\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.977962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977986    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-daemon-config\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.977998    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978054    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gljzz\" (UniqueName: \"kubernetes.io/projected/f1cb922d-534b-4fe0-8930-b115b820d656-kube-api-access-gljzz\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978097    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-run\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978120    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-host\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978153    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07ca9bba-a38c-48af-9a31-f71bad8bb876-crio-socket\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978158    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978158    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e199587-19b7-4a4b-bdd0-761b7a6e475f-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978189    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978229    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-kubelet-config\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978340    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978368    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978373    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hct8\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978408    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978443    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbzw\" (UniqueName: \"kubernetes.io/projected/984f643d-51d1-4142-b244-f8f0980a055f-kube-api-access-hxbzw\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978470    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-tmp\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978495    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-kubelet\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978524    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fba2151-22de-4a03-afcb-1d89b9c32a2b-config-volume\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978554    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978565    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1906564b-67cc-491d-b1f7-2c014046a18d-tmp\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978599    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-cnibin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978637    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978703    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:03.979427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.978780    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fba2151-22de-4a03-afcb-1d89b9c32a2b-config-volume\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:03.998956 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.998936    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kml79\" (UniqueName: \"kubernetes.io/projected/dc78343b-907d-4222-b27d-7705a2a0f19e-kube-api-access-kml79\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:03.999066 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.998963    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtrmh\" (UniqueName: \"kubernetes.io/projected/7add3842-20f6-49eb-89ad-02078f7767ef-kube-api-access-vtrmh\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:45:03.999286 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.999267    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v75s\" (UniqueName: \"kubernetes.io/projected/04a41ddb-fcb6-49bf-8468-deccd08a460c-kube-api-access-8v75s\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6\" (UID: \"04a41ddb-fcb6-49bf-8468-deccd08a460c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6"
Oct 30 10:45:03.999457 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.999441    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlsp5\" (UniqueName: \"kubernetes.io/projected/0df37afa-77d5-4bf7-bc83-0d7636453bda-kube-api-access-qlsp5\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:03.999756 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:03.999740    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn5vl\" (UniqueName: \"kubernetes.io/projected/a179c33d-aecd-4b72-b907-37f82a9aec15-kube-api-access-qn5vl\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:04.000490 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.000472    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jnkk\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:04.000581 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.000566    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zlc\" (UniqueName: \"kubernetes.io/projected/e3cc17ea-0cec-4a71-882d-2df0457dd957-kube-api-access-h8zlc\") pod \"kas-connection-checker-685895c7c9-r9ns7\" (UID: \"e3cc17ea-0cec-4a71-882d-2df0457dd957\") " pod="kube-system/kas-connection-checker-685895c7c9-r9ns7"
Oct 30 10:45:04.000830 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.000815    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qshr8\" (UniqueName: \"kubernetes.io/projected/2fba2151-22de-4a03-afcb-1d89b9c32a2b-kube-api-access-qshr8\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:04.000872 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.000857    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrsk\" (UniqueName: \"kubernetes.io/projected/eb883d09-b098-410d-bc1d-f68e24021d17-kube-api-access-phrsk\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.004174 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.004158    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq49n\" (UniqueName: \"kubernetes.io/projected/418cb483-75bb-49b0-9c67-f3b70c26c2b4-kube-api-access-hq49n\") pod \"kas-connection-checker-685895c7c9-xfp2x\" (UID: \"418cb483-75bb-49b0-9c67-f3b70c26c2b4\") " pod="kube-system/kas-connection-checker-685895c7c9-xfp2x"
Oct 30 10:45:04.004536 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.004521    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqwn6\" (UniqueName: \"kubernetes.io/projected/9e199587-19b7-4a4b-bdd0-761b7a6e475f-kube-api-access-cqwn6\") pod \"insights-operator-7c4986d449-v69fn\" (UID: \"9e199587-19b7-4a4b-bdd0-761b7a6e475f\") " pod="openshift-insights/insights-operator-7c4986d449-v69fn"
Oct 30 10:45:04.004587 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.004564    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlkjh\" (UniqueName: \"kubernetes.io/projected/1cbcd9b8-ce20-494c-9a66-475272f2a561-kube-api-access-hlkjh\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:04.004656 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.004641    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-758g5\" (UniqueName: \"kubernetes.io/projected/ded181d7-f3c4-4324-946f-2770e8e3d0fd-kube-api-access-758g5\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:45:04.009029 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.008998    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r272g\" (UniqueName: \"kubernetes.io/projected/f384be3a-0f13-4b0a-ba1b-d63d2095b9d2-kube-api-access-r272g\") pod \"kas-connection-checker-685895c7c9-8pgqv\" (UID: \"f384be3a-0f13-4b0a-ba1b-d63d2095b9d2\") " pod="kube-system/kas-connection-checker-685895c7c9-8pgqv"
Oct 30 10:45:04.009086 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.009076    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twjpp\" (UniqueName: \"kubernetes.io/projected/1114f0eb-6317-4004-88fa-55f60fc991a5-kube-api-access-twjpp\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.009201 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.009182    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:04.009275 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.009256    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42qs\" (UniqueName: \"kubernetes.io/projected/167b7d19-70a6-43c4-a2c2-4386a574d3c3-kube-api-access-b42qs\") pod \"network-check-target-ccnv5\" (UID: \"167b7d19-70a6-43c4-a2c2-4386a574d3c3\") " pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:45:04.009323 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.009309    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ncc7\" (UniqueName: \"kubernetes.io/projected/7333057e-e1db-4970-bf86-eede9e94c06a-kube-api-access-6ncc7\") pod \"service-ca-operator-544d749c8c-q8lst\" (UID: \"7333057e-e1db-4970-bf86-eede9e94c06a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst"
Oct 30 10:45:04.009840 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.009825    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpl2q\" (UniqueName: \"kubernetes.io/projected/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-api-access-lpl2q\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:04.018010 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.017971    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:04.018165 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.018145    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2fpx\" (UniqueName: \"kubernetes.io/projected/1906564b-67cc-491d-b1f7-2c014046a18d-kube-api-access-r2fpx\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.018224 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.018150    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq8tw\" (UniqueName: \"kubernetes.io/projected/d88e5148-03db-43f8-a46f-107d8d42af0f-kube-api-access-vq8tw\") pod \"ingress-canary-7kq9h\" (UID: \"d88e5148-03db-43f8-a46f-107d8d42af0f\") " pod="openshift-ingress-canary/ingress-canary-7kq9h"
Oct 30 10:45:04.018330 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.018317    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8grgp\" (UniqueName: \"kubernetes.io/projected/fa764bd7-2feb-4a11-880e-70118d1ac639-kube-api-access-8grgp\") pod \"network-check-source-6f78d59f99-w7wqz\" (UID: \"fa764bd7-2feb-4a11-880e-70118d1ac639\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-w7wqz"
Oct 30 10:45:04.018485 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.018463    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2hnd\" (UniqueName: \"kubernetes.io/projected/b8b2a817-e3d9-4361-98cd-c0a40c93b84d-kube-api-access-z2hnd\") pod \"volume-data-source-validator-58c9f5f46d-5xkvn\" (UID: \"b8b2a817-e3d9-4361-98cd-c0a40c93b84d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-5xkvn"
Oct 30 10:45:04.023216 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.023201    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hct8\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:04.023567 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.023549    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxbzw\" (UniqueName: \"kubernetes.io/projected/984f643d-51d1-4142-b244-f8f0980a055f-kube-api-access-hxbzw\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.023857 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.023843    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gljzz\" (UniqueName: \"kubernetes.io/projected/f1cb922d-534b-4fe0-8930-b115b820d656-kube-api-access-gljzz\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:45:04.024551 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.024533    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvnqf\" (UniqueName: \"kubernetes.io/projected/909137f2-6cf7-4fc8-8bcd-b0620b87aa5a-kube-api-access-tvnqf\") pod \"console-operator-7cd56b8dbc-2hmr5\" (UID: \"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:04.026532 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.026515    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54d74\" (UniqueName: \"kubernetes.io/projected/9bd59176-1f2c-4169-a6bc-45ca82e270eb-kube-api-access-54d74\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:04.079409 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079386    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-bin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.079497 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079413    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-log-socket\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.079497 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079427    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-run\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.079497 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079445    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-host\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.079497 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079467    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07ca9bba-a38c-48af-9a31-f71bad8bb876-crio-socket\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:04.079497 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079489    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-log-socket\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.079497 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079492    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-kubelet-config\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079514    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-bin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079524    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079530    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-run\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079538    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-kubelet-config\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079550    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07ca9bba-a38c-48af-9a31-f71bad8bb876-crio-socket\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079552    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-host\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079557    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079593    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-kubelet\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079622    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079643    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-kubelet\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079645    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-cnibin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079688    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-systemd-units\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079696    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-cnibin\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079714    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-systemd-units\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079713    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.079730    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.079746    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079747    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-kubernetes\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.079768 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079765    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079795    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-conf\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.079811    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.579792344 +0000 UTC m=+1.960241779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079821    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-kubernetes\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079837    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-conf-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079901    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-conf\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079914    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-conf-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079916    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysctl-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080285    4849 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.079862    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080442    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-netns\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080470    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080494    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded181d7-f3c4-4324-946f-2770e8e3d0fd-hosts-file\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080510    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bd59176-1f2c-4169-a6bc-45ca82e270eb-service-ca-bundle\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080518    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080543    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-netns\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080551    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cb922d-534b-4fe0-8930-b115b820d656-host-slash\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080565    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080574    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.080572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080582    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cb922d-534b-4fe0-8930-b115b820d656-host-slash\") pod \"iptables-alerter-2kxdq\" (UID: \"f1cb922d-534b-4fe0-8930-b115b820d656\") " pod="openshift-network-operator/iptables-alerter-2kxdq"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080587    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-var-lib-kubelet\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080590    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded181d7-f3c4-4324-946f-2770e8e3d0fd-hosts-file\") pod \"node-resolver-887wc\" (UID: \"ded181d7-f3c4-4324-946f-2770e8e3d0fd\") " pod="openshift-dns/node-resolver-887wc"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080627    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-multus\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080638    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-var-lib-kubelet\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080653    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-sys-fs\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080678    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-multus-certs\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080701    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-etc-kubernetes\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080721    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-sys-fs\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080680    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-var-lib-cni-multus\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080723    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-slash\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080742    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-multus-certs\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080759    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-etc-kubernetes\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080759    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-slash\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080767    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-var-lib-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080795    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080811    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-systemd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080827    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-var-lib-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080829    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-bin\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080851    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-sys\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080855    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-systemd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080864    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-netns\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080878    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-kubelet\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080879    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-bin\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.080894    4849 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080909    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-kubelet\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080916    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-ovn\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080949    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-netns\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.080959    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.580941648 +0000 UTC m=+1.961391085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080949    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-sys\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.080897    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-run-ovn\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081002    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-socket-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081046    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-node-log\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081072    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081099    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081125    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081132    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-node-log\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081150    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-socket-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081156    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081201    4849 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081239    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.58122833 +0000 UTC m=+1.961677747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081198    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081257    4849 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081275    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-cnibin\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081296    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.581282408 +0000 UTC m=+1.961731826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081298    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-cnibin\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:04.081313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081318    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7add3842-20f6-49eb-89ad-02078f7767ef-host\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081345    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081375    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081391    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7add3842-20f6-49eb-89ad-02078f7767ef-host\") pod \"node-ca-qxz95\" (UID: \"7add3842-20f6-49eb-89ad-02078f7767ef\") " pod="openshift-image-registry/node-ca-qxz95"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081400    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-device-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081430    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-netd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081435    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081443    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-device-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081454    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-dbus\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081475    4849 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081483    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-host-cni-netd\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081502    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc612ba8-81c6-4d10-aecf-f8efdc6fea4f-dbus\") pod \"global-pull-secret-syncer-lkmz6\" (UID: \"bc612ba8-81c6-4d10-aecf-f8efdc6fea4f\") " pod="kube-system/global-pull-secret-syncer-lkmz6"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081476    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081527    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.581513471 +0000 UTC m=+1.961962904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081541    4849 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081558    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-etc-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081574    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.581563341 +0000 UTC m=+1.962012774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : secret "metrics-daemon-secret" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081582    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb883d09-b098-410d-bc1d-f68e24021d17-etc-openvswitch\") pod \"ovnkube-node-5sl2r\" (UID: \"eb883d09-b098-410d-bc1d-f68e24021d17\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081587    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07ca9bba-a38c-48af-9a31-f71bad8bb876-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081599    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-k8s-cni-cncf-io\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081624    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081649    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-socket-dir-parent\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081651    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-host-run-k8s-cni-cncf-io\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081675    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081694    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-socket-dir-parent\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081698    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-modprobe-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081728    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081730    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-lib-modules\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081736    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081744    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-system-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081697    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.081770    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.581760732 +0000 UTC m=+1.962210152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081792    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-registration-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081795    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-modprobe-d\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081800    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-system-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081819    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081826    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/984f643d-51d1-4142-b244-f8f0980a055f-registration-dir\") pod \"aws-ebs-csi-driver-node-fwtxz\" (UID: \"984f643d-51d1-4142-b244-f8f0980a055f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwtxz"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081839    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-lib-modules\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081842    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysconfig\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081866    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-hostroot\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081867    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc78343b-907d-4222-b27d-7705a2a0f19e-system-cni-dir\") pod \"multus-additional-cni-plugins-4k458\" (UID: \"dc78343b-907d-4222-b27d-7705a2a0f19e\") " pod="openshift-multus/multus-additional-cni-plugins-4k458"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081883    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-sysconfig\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081894    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081924    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-systemd\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081938    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-multus-cni-dir\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081896    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1114f0eb-6317-4004-88fa-55f60fc991a5-hostroot\") pod \"multus-2c8hg\" (UID: \"1114f0eb-6317-4004-88fa-55f60fc991a5\") " pod="openshift-multus/multus-2c8hg"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081952    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.081995    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1906564b-67cc-491d-b1f7-2c014046a18d-etc-systemd\") pod \"tuned-rnv69\" (UID: \"1906564b-67cc-491d-b1f7-2c014046a18d\") " pod="openshift-cluster-node-tuning-operator/tuned-rnv69"
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.082041    4849 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:04.082666 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.082080    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:04.582071133 +0000 UTC m=+1.962520574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:04.084093 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.082765    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1cbcd9b8-ce20-494c-9a66-475272f2a561-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lw745\" (UID: \"1cbcd9b8-ce20-494c-9a66-475272f2a561\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:04.154613 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.154594    4849 scope.go:122] "RemoveContainer" containerID="f8912bf9c29e99061121aba6a3b280560ad379cf25a59577044eb706d3d8e5f9"
Oct 30 10:45:04.154745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.154728    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"
Oct 30 10:45:04.476360 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.476297    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lw745"]
Oct 30 10:45:04.479570 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:04.479546    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbcd9b8_ce20_494c_9a66_475272f2a561.slice/crio-315b06c5fcfc812cd04ebc678a5a314fe9af00a9db3e96dca195b954dfabbb39 WatchSource:0}: Error finding container 315b06c5fcfc812cd04ebc678a5a314fe9af00a9db3e96dca195b954dfabbb39: Status 404 returned error can't find the container with id 315b06c5fcfc812cd04ebc678a5a314fe9af00a9db3e96dca195b954dfabbb39
Oct 30 10:45:04.481485 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.481470    4849 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 10:45:04.586924 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.586897    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:04.587040 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.586940    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:04.587040 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.586959    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:04.587040 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.586978    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:04.587040 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.586999    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:04.587040 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.587041    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.587074    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.587116    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587076    4849 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587171    4849 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587076    4849 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587214    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587225    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587083    4849 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587091    4849 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:45:04.587277 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587135    4849 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587140    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587316    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587213    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587191494 +0000 UTC m=+2.967640928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587348    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587336006 +0000 UTC m=+2.967785426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : secret "metrics-daemon-secret" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587358    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587352832 +0000 UTC m=+2.967802252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587373    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587367888 +0000 UTC m=+2.967817304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587384    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587380272 +0000 UTC m=+2.967829689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587396    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587392283 +0000 UTC m=+2.967841700 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587408    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587404232 +0000 UTC m=+2.967853648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:04.587737 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.587418    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:05.587414549 +0000 UTC m=+2.967863965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:45:04.803571 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.803517    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745" event={"ID":"1cbcd9b8-ce20-494c-9a66-475272f2a561","Type":"ContainerStarted","Data":"315b06c5fcfc812cd04ebc678a5a314fe9af00a9db3e96dca195b954dfabbb39"}
Oct 30 10:45:04.805061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.805045    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 10:45:04.805452 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.805439    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/1.log"
Oct 30 10:45:04.805501 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.805468    4849 generic.go:356] "Generic (PLEG): container finished" podID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" containerID="2056d55f406eb39adafaef142d43896873f407a40721a862797f71ec4e3db1e8" exitCode=255
Oct 30 10:45:04.805562 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.805544    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" event={"ID":"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a","Type":"ContainerDied","Data":"2056d55f406eb39adafaef142d43896873f407a40721a862797f71ec4e3db1e8"}
Oct 30 10:45:04.805629 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.805591    4849 scope.go:122] "RemoveContainer" containerID="f8912bf9c29e99061121aba6a3b280560ad379cf25a59577044eb706d3d8e5f9"
Oct 30 10:45:04.805982 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:04.805944    4849 scope.go:122] "RemoveContainer" containerID="2056d55f406eb39adafaef142d43896873f407a40721a862797f71ec4e3db1e8"
Oct 30 10:45:04.806114 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:04.806096    4849 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2hmr5_openshift-console-operator(909137f2-6cf7-4fc8-8bcd-b0620b87aa5a)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" podUID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a"
Oct 30 10:45:05.596295 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596266    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596305    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596334    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596361    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596390    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596413    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596435    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596419    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596470    4849 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596487    4849 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596490    4849 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596494    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596478251 +0000 UTC m=+4.976927685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596483    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596494    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596544    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596529265 +0000 UTC m=+4.976978686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596547    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:45:05.596550 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596558    4849 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596561    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls podName:2fba2151-22de-4a03-afcb-1d89b9c32a2b nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596553639 +0000 UTC m=+4.977003056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls") pod "dns-default-mx9gc" (UID: "2fba2151-22de-4a03-afcb-1d89b9c32a2b") : secret "dns-default-metrics-tls" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596577    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596571617 +0000 UTC m=+4.977021033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596411    4849 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596605    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596586606 +0000 UTC m=+4.977036025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596630    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596614724 +0000 UTC m=+4.977064150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596649    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs podName:0df37afa-77d5-4bf7-bc83-0d7636453bda nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596640848 +0000 UTC m=+4.977090270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs") pod "network-metrics-daemon-hr78s" (UID: "0df37afa-77d5-4bf7-bc83-0d7636453bda") : secret "metrics-daemon-secret" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.596675    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596759    4849 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:45:05.597095 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:05.596797    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:07.596786299 +0000 UTC m=+4.977235716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:45:05.810887 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:05.810857    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 10:45:06.446165 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:06.446099    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-887wc_ded181d7-f3c4-4324-946f-2770e8e3d0fd/dns-node-resolver/0.log"
Oct 30 10:45:06.815524 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:06.815486    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lw745" event={"ID":"1cbcd9b8-ce20-494c-9a66-475272f2a561","Type":"ContainerStarted","Data":"263a0f4d8d6b38db205b3e2341c76de55a005b1103dda27247f4f4b16ff277f1"}
Oct 30 10:45:06.882776 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:06.882750    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:06.883172 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:06.883156    4849 scope.go:122] "RemoveContainer" containerID="2056d55f406eb39adafaef142d43896873f407a40721a862797f71ec4e3db1e8"
Oct 30 10:45:06.883326 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:06.883309    4849 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2hmr5_openshift-console-operator(909137f2-6cf7-4fc8-8bcd-b0620b87aa5a)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" podUID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a"
Oct 30 10:45:07.613920 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.613875    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.613932    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.613969    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.613997    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.614016    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.614052    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614058    4849 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.614082    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:07.614138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.614120    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614180    4849 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614184    4849 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614183    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls podName:07ca9bba-a38c-48af-9a31-f71bad8bb876 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:11.614164967 +0000 UTC m=+8.994614391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2kxlm" (UID: "07ca9bba-a38c-48af-9a31-f71bad8bb876") : secret "insights-runtime-extractor-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614180    4849 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614259    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:11.614245053 +0000 UTC m=+8.994694482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614235    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614291    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bdbf894c-qszt7: secret "image-registry-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614248    4849 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614328    4849 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-794b85596b-gvxrs: secret "image-registry-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614277    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls podName:a179c33d-aecd-4b72-b907-37f82a9aec15 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:11.614268046 +0000 UTC m=+8.994717504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h78lt" (UID: "a179c33d-aecd-4b72-b907-37f82a9aec15") : secret "samples-operator-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614381    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs podName:9bd59176-1f2c-4169-a6bc-45ca82e270eb nodeName:}" failed. No retries permitted until 2026-10-30 10:45:11.614357201 +0000 UTC m=+8.994806623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs") pod "router-default-65668b76cf-ckhgx" (UID: "9bd59176-1f2c-4169-a6bc-45ca82e270eb") : secret "router-metrics-certs-default" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614408    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls podName:9fb55617-3df2-401d-876b-2fc95f902c53 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:11.61439804 +0000 UTC m=+8.994847463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls") pod "image-registry-67bdbf894c-qszt7" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53") : secret "image-registry-tls" not found
Oct 30 10:45:07.614534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:07.614431    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls podName:6fcbfeec-799d-4118-a2cd-2fe79a2a01a5 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:11.614422589 +0000 UTC m=+8.994872012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls") pod "image-registry-794b85596b-gvxrs" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5") : secret "image-registry-tls" not found
Oct 30 10:45:07.616740 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.616712    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0df37afa-77d5-4bf7-bc83-0d7636453bda-metrics-certs\") pod \"network-metrics-daemon-hr78s\" (UID: \"0df37afa-77d5-4bf7-bc83-0d7636453bda\") " pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:07.616865 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.616749    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fba2151-22de-4a03-afcb-1d89b9c32a2b-metrics-tls\") pod \"dns-default-mx9gc\" (UID: \"2fba2151-22de-4a03-afcb-1d89b9c32a2b\") " pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:07.645626 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.645606    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxz95_7add3842-20f6-49eb-89ad-02078f7767ef/node-ca/0.log"
Oct 30 10:45:07.759937 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.759908    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hr78s"
Oct 30 10:45:07.768064 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:07.768035    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:08.064765 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.064741    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hr78s"]
Oct 30 10:45:08.070005 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:08.069982    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df37afa_77d5_4bf7_bc83_0d7636453bda.slice/crio-097abac8cb9741b4fb3c43e07ed0900c5cbbdac3c825a8d460ae763e068282e5 WatchSource:0}: Error finding container 097abac8cb9741b4fb3c43e07ed0900c5cbbdac3c825a8d460ae763e068282e5: Status 404 returned error can't find the container with id 097abac8cb9741b4fb3c43e07ed0900c5cbbdac3c825a8d460ae763e068282e5
Oct 30 10:45:08.080160 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.080134    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mx9gc"]
Oct 30 10:45:08.084374 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:08.084353    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fba2151_22de_4a03_afcb_1d89b9c32a2b.slice/crio-309d43a895ac07cf3b4bf4786c0f0fb89b0571ae030dfc673b96e48db33396c6 WatchSource:0}: Error finding container 309d43a895ac07cf3b4bf4786c0f0fb89b0571ae030dfc673b96e48db33396c6: Status 404 returned error can't find the container with id 309d43a895ac07cf3b4bf4786c0f0fb89b0571ae030dfc673b96e48db33396c6
Oct 30 10:45:08.248544 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.248525    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7kq9h_d88e5148-03db-43f8-a46f-107d8d42af0f/serve-healthcheck-canary/0.log"
Oct 30 10:45:08.647503 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.647479    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6_04a41ddb-fcb6-49bf-8468-deccd08a460c/kube-storage-version-migrator-operator/0.log"
Oct 30 10:45:08.828887 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.828859    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr78s" event={"ID":"0df37afa-77d5-4bf7-bc83-0d7636453bda","Type":"ContainerStarted","Data":"097abac8cb9741b4fb3c43e07ed0900c5cbbdac3c825a8d460ae763e068282e5"}
Oct 30 10:45:08.830278 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.830240    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mx9gc" event={"ID":"2fba2151-22de-4a03-afcb-1d89b9c32a2b","Type":"ContainerStarted","Data":"309d43a895ac07cf3b4bf4786c0f0fb89b0571ae030dfc673b96e48db33396c6"}
Oct 30 10:45:08.831864 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.831842    4849 generic.go:356] "Generic (PLEG): container finished" podID="9e199587-19b7-4a4b-bdd0-761b7a6e475f" containerID="c72186ae410aa7a8ddf608b6ce5fa204dad865cf52b9b90bd9ac91862be54c05" exitCode=0
Oct 30 10:45:08.831993 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.831876    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-v69fn" event={"ID":"9e199587-19b7-4a4b-bdd0-761b7a6e475f","Type":"ContainerDied","Data":"c72186ae410aa7a8ddf608b6ce5fa204dad865cf52b9b90bd9ac91862be54c05"}
Oct 30 10:45:08.832259 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.832239    4849 scope.go:122] "RemoveContainer" containerID="c72186ae410aa7a8ddf608b6ce5fa204dad865cf52b9b90bd9ac91862be54c05"
Oct 30 10:45:08.847174 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.847155    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q6ddt"
Oct 30 10:45:08.951325 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:08.949994    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:09.838286 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:09.838257    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr78s" event={"ID":"0df37afa-77d5-4bf7-bc83-0d7636453bda","Type":"ContainerStarted","Data":"c5d82276a406f39632bb55902f0b1ba336e0c0d534ae330d03ba87de83b4977f"}
Oct 30 10:45:09.838683 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:09.838319    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hr78s" event={"ID":"0df37afa-77d5-4bf7-bc83-0d7636453bda","Type":"ContainerStarted","Data":"a5ad6ece4c0fd5db38a9a8d5b15723d7499f44406ef7e7de9708997bc6f9427c"}
Oct 30 10:45:09.840755 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:09.840629    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mx9gc" event={"ID":"2fba2151-22de-4a03-afcb-1d89b9c32a2b","Type":"ContainerStarted","Data":"d8c6f91b139f6bce7b5f7c50b166158b1a826c22117756ef7ed21c1cab215d11"}
Oct 30 10:45:09.843071 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:09.843047    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-v69fn" event={"ID":"9e199587-19b7-4a4b-bdd0-761b7a6e475f","Type":"ContainerStarted","Data":"ba0045341ed615092548f2397eecf7cad2146add9835e463622087f0b8bba617"}
Oct 30 10:45:09.893192 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:09.893172    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:45:10.848544 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:10.848508    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mx9gc" event={"ID":"2fba2151-22de-4a03-afcb-1d89b9c32a2b","Type":"ContainerStarted","Data":"3cfabd64e6880168878c1047eebb2d00afcf9879d975e38af75832620c81dbb2"}
Oct 30 10:45:11.650772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.650738    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:11.650772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.650775    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:11.650982 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.650796    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:11.650982 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:11.650857    4849 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 10:45:11.650982 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:11.650917    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert podName:3138c05b-7ff7-463e-9d91-ea7fab54a541 nodeName:}" failed. No retries permitted until 2026-10-30 10:45:19.650903071 +0000 UTC m=+17.031352566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-fdv8p" (UID: "3138c05b-7ff7-463e-9d91-ea7fab54a541") : secret "networking-console-plugin-cert" not found
Oct 30 10:45:11.650982 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.650918    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:11.651193 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.650992    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:11.651193 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.651096    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:11.653924 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.653892    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a179c33d-aecd-4b72-b907-37f82a9aec15-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h78lt\" (UID: \"a179c33d-aecd-4b72-b907-37f82a9aec15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:11.654102 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.653937    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bd59176-1f2c-4169-a6bc-45ca82e270eb-metrics-certs\") pod \"router-default-65668b76cf-ckhgx\" (UID: \"9bd59176-1f2c-4169-a6bc-45ca82e270eb\") " pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:11.654102 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.653975    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"image-registry-67bdbf894c-qszt7\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") " pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:11.654102 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.654000    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07ca9bba-a38c-48af-9a31-f71bad8bb876-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2kxlm\" (UID: \"07ca9bba-a38c-48af-9a31-f71bad8bb876\") " pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:11.654277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.654120    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"image-registry-794b85596b-gvxrs\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") " pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:11.655910 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.655887    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:11.656957 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.656932    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"
Oct 30 10:45:11.658065 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.658046    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:11.670856 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.670829    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:11.670856 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.670856    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2kxlm"
Oct 30 10:45:11.852595 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:11.852573    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:12.016619 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.016558    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65668b76cf-ckhgx"]
Oct 30 10:45:12.020269 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:12.020243    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd59176_1f2c_4169_a6bc_45ca82e270eb.slice/crio-da7e8dd559331ee459da9b8f398f672b50adf257dc9b67126fb345a571d93080 WatchSource:0}: Error finding container da7e8dd559331ee459da9b8f398f672b50adf257dc9b67126fb345a571d93080: Status 404 returned error can't find the container with id da7e8dd559331ee459da9b8f398f672b50adf257dc9b67126fb345a571d93080
Oct 30 10:45:12.020796 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:12.020767    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb55617_3df2_401d_876b_2fc95f902c53.slice/crio-f6f09e278bcade10ef9ef3e2081baaef74b3a33a189a70a6d5e31fb505193d0a WatchSource:0}: Error finding container f6f09e278bcade10ef9ef3e2081baaef74b3a33a189a70a6d5e31fb505193d0a: Status 404 returned error can't find the container with id f6f09e278bcade10ef9ef3e2081baaef74b3a33a189a70a6d5e31fb505193d0a
Oct 30 10:45:12.025726 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.025701    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67bdbf894c-qszt7"]
Oct 30 10:45:12.026354 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.026324    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt"]
Oct 30 10:45:12.029902 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.029877    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-794b85596b-gvxrs"]
Oct 30 10:45:12.033404 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:12.033382    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fcbfeec_799d_4118_a2cd_2fe79a2a01a5.slice/crio-4c7e3e3d3eeba4f46065e95e9a436e08b11911ed94db028abf80dc950f25eb66 WatchSource:0}: Error finding container 4c7e3e3d3eeba4f46065e95e9a436e08b11911ed94db028abf80dc950f25eb66: Status 404 returned error can't find the container with id 4c7e3e3d3eeba4f46065e95e9a436e08b11911ed94db028abf80dc950f25eb66
Oct 30 10:45:12.046150 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.046131    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2kxlm"]
Oct 30 10:45:12.049457 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:12.049437    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ca9bba_a38c_48af_9a31_f71bad8bb876.slice/crio-aa8e284ced88ef1f983668d8ffbf9a7634b0d76552b4ddd49e0f1f193328830d WatchSource:0}: Error finding container aa8e284ced88ef1f983668d8ffbf9a7634b0d76552b4ddd49e0f1f193328830d: Status 404 returned error can't find the container with id aa8e284ced88ef1f983668d8ffbf9a7634b0d76552b4ddd49e0f1f193328830d
Oct 30 10:45:12.444975 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.444944    4849 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:12.445427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.445410    4849 scope.go:122] "RemoveContainer" containerID="2056d55f406eb39adafaef142d43896873f407a40721a862797f71ec4e3db1e8"
Oct 30 10:45:12.445654 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:12.445634    4849 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2hmr5_openshift-console-operator(909137f2-6cf7-4fc8-8bcd-b0620b87aa5a)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" podUID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a"
Oct 30 10:45:12.856890 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.856858    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-794b85596b-gvxrs" event={"ID":"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5","Type":"ContainerStarted","Data":"06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee"}
Oct 30 10:45:12.857327 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.856944    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-794b85596b-gvxrs" event={"ID":"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5","Type":"ContainerStarted","Data":"4c7e3e3d3eeba4f46065e95e9a436e08b11911ed94db028abf80dc950f25eb66"}
Oct 30 10:45:12.858602 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.858574    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7" event={"ID":"9fb55617-3df2-401d-876b-2fc95f902c53","Type":"ContainerStarted","Data":"866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2"}
Oct 30 10:45:12.858726 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.858616    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7" event={"ID":"9fb55617-3df2-401d-876b-2fc95f902c53","Type":"ContainerStarted","Data":"f6f09e278bcade10ef9ef3e2081baaef74b3a33a189a70a6d5e31fb505193d0a"}
Oct 30 10:45:12.858726 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.858659    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:12.860462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.860441    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxlm" event={"ID":"07ca9bba-a38c-48af-9a31-f71bad8bb876","Type":"ContainerStarted","Data":"6f0db13b2f4a5438379ddf7afba7e2a725048fcb3eda9ca0c6dd7d49746779f0"}
Oct 30 10:45:12.860564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.860485    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxlm" event={"ID":"07ca9bba-a38c-48af-9a31-f71bad8bb876","Type":"ContainerStarted","Data":"b0727084eaebcde77432951d33c9512b6401331c3554c968b0aab50fe87485e1"}
Oct 30 10:45:12.860564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.860515    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxlm" event={"ID":"07ca9bba-a38c-48af-9a31-f71bad8bb876","Type":"ContainerStarted","Data":"aa8e284ced88ef1f983668d8ffbf9a7634b0d76552b4ddd49e0f1f193328830d"}
Oct 30 10:45:12.861601 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.861581    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt" event={"ID":"a179c33d-aecd-4b72-b907-37f82a9aec15","Type":"ContainerStarted","Data":"a88381bd0827ed21149b6b86d0232e265bc5abee5183de91c3a1be679fb075f8"}
Oct 30 10:45:12.863183 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.863154    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65668b76cf-ckhgx" event={"ID":"9bd59176-1f2c-4169-a6bc-45ca82e270eb","Type":"ContainerStarted","Data":"a34cbb5f9296d8e718a06dc5e3b39258366142887a1f65e68e1acb146319007a"}
Oct 30 10:45:12.863275 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:12.863196    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65668b76cf-ckhgx" event={"ID":"9bd59176-1f2c-4169-a6bc-45ca82e270eb","Type":"ContainerStarted","Data":"da7e8dd559331ee459da9b8f398f672b50adf257dc9b67126fb345a571d93080"}
Oct 30 10:45:13.445949 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:13.445919    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:13.446200 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:13.446180    4849 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 10:45:13.658594 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:13.658568    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:13.661644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:13.661622    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:13.870424 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:13.869912    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt" event={"ID":"a179c33d-aecd-4b72-b907-37f82a9aec15","Type":"ContainerStarted","Data":"f4275579adb7f13fd0cd89475262cbd07dc3895d5edb1ad65877b5e88adaf5b0"}
Oct 30 10:45:13.870424 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:13.869952    4849 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 10:45:13.870424 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:13.869982    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h78lt" event={"ID":"a179c33d-aecd-4b72-b907-37f82a9aec15","Type":"ContainerStarted","Data":"8f281fcaf4c63d143973f0f93f54dba98b4cd578c1dbb1a3e09442c44da6e4c4"}
Oct 30 10:45:14.234061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:14.233986    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sl2r"
Oct 30 10:45:14.878241 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:14.878212    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2kxlm" event={"ID":"07ca9bba-a38c-48af-9a31-f71bad8bb876","Type":"ContainerStarted","Data":"ec891d0cc1c2547b933f471adb61bc214bdd35ded6203f9fda4bc48c27a93efc"}
Oct 30 10:45:14.878241 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:14.878239    4849 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 10:45:15.747980 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:15.747953    4849 scope.go:122] "RemoveContainer" containerID="2056d55f406eb39adafaef142d43896873f407a40721a862797f71ec4e3db1e8"
Oct 30 10:45:15.883093 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:15.883071    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 10:45:15.883491 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:15.883183    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" event={"ID":"909137f2-6cf7-4fc8-8bcd-b0620b87aa5a","Type":"ContainerStarted","Data":"ad24d4660b580a6405b9a74828919a1ca173f08c450d1b43841032d0ee05f01a"}
Oct 30 10:45:15.883623 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:15.883599    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:15.884473 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:15.884450    4849 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-2hmr5 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body=
Oct 30 10:45:15.884579 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:15.884503    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5" podUID="909137f2-6cf7-4fc8-8bcd-b0620b87aa5a" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused"
Oct 30 10:45:16.895756 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:16.895728    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2hmr5"
Oct 30 10:45:16.982449 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:16.982418    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:18.554662 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:18.554620    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:18.555224 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:18.554783    4849 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 10:45:18.556093 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:18.556063    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-65668b76cf-ckhgx"
Oct 30 10:45:19.715097 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:19.715060    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:19.717537 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:19.717508    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3138c05b-7ff7-463e-9d91-ea7fab54a541-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-fdv8p\" (UID: \"3138c05b-7ff7-463e-9d91-ea7fab54a541\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:19.757772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:19.757745    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"
Oct 30 10:45:20.059965 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:20.059937    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-fdv8p"]
Oct 30 10:45:20.062735 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:20.062702    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3138c05b_7ff7_463e_9d91_ea7fab54a541.slice/crio-097eb64b0b99a33ae5d4a2d89f8d6d4f6101e0771faae09b154ef188f6bfd4d0 WatchSource:0}: Error finding container 097eb64b0b99a33ae5d4a2d89f8d6d4f6101e0771faae09b154ef188f6bfd4d0: Status 404 returned error can't find the container with id 097eb64b0b99a33ae5d4a2d89f8d6d4f6101e0771faae09b154ef188f6bfd4d0
Oct 30 10:45:20.476955 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:20.476884    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67bdbf894c-qszt7"]
Oct 30 10:45:20.906736 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:20.906692    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p" event={"ID":"3138c05b-7ff7-463e-9d91-ea7fab54a541","Type":"ContainerStarted","Data":"097eb64b0b99a33ae5d4a2d89f8d6d4f6101e0771faae09b154ef188f6bfd4d0"}
Oct 30 10:45:21.865784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:21.865756    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mx9gc"
Oct 30 10:45:21.912115 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:21.912085    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-fdv8p" event={"ID":"3138c05b-7ff7-463e-9d91-ea7fab54a541","Type":"ContainerStarted","Data":"2c401451c5d1ccab211a917c3022d85bdeb7ff8170e2d6ebf40a2a89548c0739"}
Oct 30 10:45:23.235522 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:23.235480    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mx9gc_2fba2151-22de-4a03-afcb-1d89b9c32a2b/dns/0.log"
Oct 30 10:45:23.436706 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:23.436671    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mx9gc_2fba2151-22de-4a03-afcb-1d89b9c32a2b/kube-rbac-proxy/0.log"
Oct 30 10:45:24.036330 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:24.036301    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-887wc_ded181d7-f3c4-4324-946f-2770e8e3d0fd/dns-node-resolver/0.log"
Oct 30 10:45:24.831799 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:24.831773    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67bdbf894c-qszt7_9fb55617-3df2-401d-876b-2fc95f902c53/registry/0.log"
Oct 30 10:45:25.030562 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:25.030529    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-794b85596b-gvxrs_6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/registry/0.log"
Oct 30 10:45:25.433817 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:25.433789    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxz95_7add3842-20f6-49eb-89ad-02078f7767ef/node-ca/0.log"
Oct 30 10:45:25.830747 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:25.830720    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65668b76cf-ckhgx_9bd59176-1f2c-4169-a6bc-45ca82e270eb/router/0.log"
Oct 30 10:45:26.029706 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:26.029685    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7kq9h_d88e5148-03db-43f8-a46f-107d8d42af0f/serve-healthcheck-canary/0.log"
Oct 30 10:45:27.031017 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:27.030988    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6_04a41ddb-fcb6-49bf-8468-deccd08a460c/kube-storage-version-migrator-operator/0.log"
Oct 30 10:45:28.825093 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.825063    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dj2v9"]
Oct 30 10:45:28.867251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.867228    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.872728 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872707    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-wtmp\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.872859 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872735    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-tls\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.872859 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872795    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.872859 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872819    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.872859 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872839    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-textfile\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.873010 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872862    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2686d713-b3ff-4f5a-8183-adff6efb48d0-metrics-client-ca\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.873010 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872883    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-root\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.873010 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.872977    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dgq8\" (UniqueName: \"kubernetes.io/projected/2686d713-b3ff-4f5a-8183-adff6efb48d0-kube-api-access-7dgq8\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.873010 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.873004    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-sys\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.897483 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.897461    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 10:45:28.897483 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.897470    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 10:45:28.897648 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.897525    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 10:45:28.897648 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.897566    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jgh82\""
Oct 30 10:45:28.897648 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.897640    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 10:45:28.974143 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974121    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2686d713-b3ff-4f5a-8183-adff6efb48d0-metrics-client-ca\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974152    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-root\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974170    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dgq8\" (UniqueName: \"kubernetes.io/projected/2686d713-b3ff-4f5a-8183-adff6efb48d0-kube-api-access-7dgq8\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974194    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-sys\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974221    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-root\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974251 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974236    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-wtmp\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974268    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-tls\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974297    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974331    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974354    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-wtmp\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974362    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-textfile\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974450    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2686d713-b3ff-4f5a-8183-adff6efb48d0-sys\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974740 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974653    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-textfile\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974740 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974687    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2686d713-b3ff-4f5a-8183-adff6efb48d0-metrics-client-ca\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.974907 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.974886    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.976430 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.976411    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-tls\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:28.976581 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:28.976559    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2686d713-b3ff-4f5a-8183-adff6efb48d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:29.000325 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:29.000300    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dgq8\" (UniqueName: \"kubernetes.io/projected/2686d713-b3ff-4f5a-8183-adff6efb48d0-kube-api-access-7dgq8\") pod \"node-exporter-dj2v9\" (UID: \"2686d713-b3ff-4f5a-8183-adff6efb48d0\") " pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:29.218348 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:29.218296    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dj2v9"
Oct 30 10:45:29.248292 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:45:29.248268    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2686d713_b3ff_4f5a_8183_adff6efb48d0.slice/crio-0e3ef532f1133f2371f189b0b7f5840ac7e833a01516e7b5b3fd187583782971 WatchSource:0}: Error finding container 0e3ef532f1133f2371f189b0b7f5840ac7e833a01516e7b5b3fd187583782971: Status 404 returned error can't find the container with id 0e3ef532f1133f2371f189b0b7f5840ac7e833a01516e7b5b3fd187583782971
Oct 30 10:45:29.950189 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:29.950154    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dj2v9" event={"ID":"2686d713-b3ff-4f5a-8183-adff6efb48d0","Type":"ContainerStarted","Data":"0e3ef532f1133f2371f189b0b7f5840ac7e833a01516e7b5b3fd187583782971"}
Oct 30 10:45:30.482220 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:30.482192    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:30.957446 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:30.957411    4849 generic.go:356] "Generic (PLEG): container finished" podID="2686d713-b3ff-4f5a-8183-adff6efb48d0" containerID="c56dd7950da60fc88cf52bf05861835498d307a9e2b4b537d4009a917ba6990d" exitCode=0
Oct 30 10:45:30.957831 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:30.957490    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dj2v9" event={"ID":"2686d713-b3ff-4f5a-8183-adff6efb48d0","Type":"ContainerDied","Data":"c56dd7950da60fc88cf52bf05861835498d307a9e2b4b537d4009a917ba6990d"}
Oct 30 10:45:31.963706 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:31.963674    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dj2v9" event={"ID":"2686d713-b3ff-4f5a-8183-adff6efb48d0","Type":"ContainerStarted","Data":"3d2a884b4d4f0805689fb606f4dedf1f9135270b232dddbe9b49e4fc2eff6f0b"}
Oct 30 10:45:31.964102 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:31.963727    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dj2v9" event={"ID":"2686d713-b3ff-4f5a-8183-adff6efb48d0","Type":"ContainerStarted","Data":"881c49e5c70d5855b6a06f5331754f579a5b668b65d9ba0d55a290190512b65a"}
Oct 30 10:45:32.030059 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:32.029988    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dj2v9" podStartSLOduration=2.658007356 podStartE2EDuration="4.02997448s" totalImagesPullingTime="1.293801657s" totalInitContainerRuntime="78.165467ms" isStatefulPod=true podCreationTimestamp="2026-10-30 10:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:45:32.025081976 +0000 UTC m=+29.405531414" watchObservedRunningTime="2026-10-30 10:45:32.02997448 +0000 UTC m=+29.410423943"
Oct 30 10:45:36.987690 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:36.987661    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:45:39.432233 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:39.432190    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:45:39.896158 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:39.896129    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ccnv5"
Oct 30 10:45:43.835243 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:43.834145    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-794b85596b-gvxrs"]
Oct 30 10:45:45.497356 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.497253    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7" podUID="9fb55617-3df2-401d-876b-2fc95f902c53" containerName="registry" containerID="cri-o://866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2" gracePeriod=30
Oct 30 10:45:45.739477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.739450    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:45.793332 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793277    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793332 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793316    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793493 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793340    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793493 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793360    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793493 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793384    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793493 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793401    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793493 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793429    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793493 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793452    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\") pod \"9fb55617-3df2-401d-876b-2fc95f902c53\" (UID: \"9fb55617-3df2-401d-876b-2fc95f902c53\") "
Oct 30 10:45:45.793885 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793851    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:45:45.794044 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.793997    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:45:45.796064 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.796013    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "kube-api-access-6jnkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:45:45.796203 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.796178    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:45:45.796274 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.796181    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:45:45.796332 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.796270    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:45:45.796332 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.796308    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:45:45.800767 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.800709    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted" pod "9fb55617-3df2-401d-876b-2fc95f902c53" (UID: "9fb55617-3df2-401d-876b-2fc95f902c53"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:45:45.894543 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894517    4849 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-trusted-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:45.894543 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894542    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jnkk\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-kube-api-access-6jnkk\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:45.894684 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894552    4849 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-registry-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:45.894684 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894561    4849 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9fb55617-3df2-401d-876b-2fc95f902c53-bound-sa-token\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:45.894684 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894569    4849 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9fb55617-3df2-401d-876b-2fc95f902c53-ca-trust-extracted\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:45.894684 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894577    4849 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-installation-pull-secrets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:45.894684 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894585    4849 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9fb55617-3df2-401d-876b-2fc95f902c53-registry-certificates\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:45.894684 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:45.894595    4849 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9fb55617-3df2-401d-876b-2fc95f902c53-image-registry-private-configuration\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:45:46.025858 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.025826    4849 generic.go:356] "Generic (PLEG): container finished" podID="9fb55617-3df2-401d-876b-2fc95f902c53" containerID="866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2" exitCode=0
Oct 30 10:45:46.025954 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.025879    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7"
Oct 30 10:45:46.025954 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.025909    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7" event={"ID":"9fb55617-3df2-401d-876b-2fc95f902c53","Type":"ContainerDied","Data":"866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2"}
Oct 30 10:45:46.026055 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.025962    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bdbf894c-qszt7" event={"ID":"9fb55617-3df2-401d-876b-2fc95f902c53","Type":"ContainerDied","Data":"f6f09e278bcade10ef9ef3e2081baaef74b3a33a189a70a6d5e31fb505193d0a"}
Oct 30 10:45:46.026055 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.025985    4849 scope.go:122] "RemoveContainer" containerID="866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2"
Oct 30 10:45:46.034350 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.034326    4849 scope.go:122] "RemoveContainer" containerID="866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2"
Oct 30 10:45:46.034642 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:45:46.034614    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2\": container with ID starting with 866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2 not found: ID does not exist" containerID="866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2"
Oct 30 10:45:46.034694 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.034652    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2"} err="failed to get container status \"866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2\": rpc error: code = NotFound desc = could not find container \"866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2\": container with ID starting with 866034ff0ef15c9b35a895307e112904fc959671a2e51847f915aed00f693da2 not found: ID does not exist"
Oct 30 10:45:46.081388 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.081366    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67bdbf894c-qszt7"]
Oct 30 10:45:46.095032 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.094999    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67bdbf894c-qszt7"]
Oct 30 10:45:46.752292 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:45:46.752264    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fb55617-3df2-401d-876b-2fc95f902c53" path="/var/lib/kubelet/pods/9fb55617-3df2-401d-876b-2fc95f902c53/volumes"
Oct 30 10:46:08.865528 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:08.865463    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-794b85596b-gvxrs" podUID="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" containerName="registry" containerID="cri-o://06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee" gracePeriod=30
Oct 30 10:46:09.101039 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.101006    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:46:09.115278 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.115252    4849 generic.go:356] "Generic (PLEG): container finished" podID="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" containerID="06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee" exitCode=0
Oct 30 10:46:09.115381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.115285    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-794b85596b-gvxrs"
Oct 30 10:46:09.115381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.115331    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-794b85596b-gvxrs" event={"ID":"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5","Type":"ContainerDied","Data":"06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee"}
Oct 30 10:46:09.115461 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.115382    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-794b85596b-gvxrs" event={"ID":"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5","Type":"ContainerDied","Data":"4c7e3e3d3eeba4f46065e95e9a436e08b11911ed94db028abf80dc950f25eb66"}
Oct 30 10:46:09.115461 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.115404    4849 scope.go:122] "RemoveContainer" containerID="06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee"
Oct 30 10:46:09.122832 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.122813    4849 scope.go:122] "RemoveContainer" containerID="06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee"
Oct 30 10:46:09.123121 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:46:09.123103    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee\": container with ID starting with 06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee not found: ID does not exist" containerID="06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee"
Oct 30 10:46:09.123167 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.123130    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee"} err="failed to get container status \"06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee\": rpc error: code = NotFound desc = could not find container \"06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee\": container with ID starting with 06ed19bf3e374328acc12a96bdc1b591185897cea0ab62b1e25b43ff3277afee not found: ID does not exist"
Oct 30 10:46:09.150953 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.150936    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151045 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.150965    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151045 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.150985    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151147 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151129    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151198 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151174    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151282 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151207    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151282 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151237    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151388 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151287    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\") pod \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\" (UID: \"6fcbfeec-799d-4118-a2cd-2fe79a2a01a5\") "
Oct 30 10:46:09.151388 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151361    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:46:09.151683 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151660    4849 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-certificates\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.151771 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.151660    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:46:09.153270 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.153248    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:46:09.153418 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.153388    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:46:09.153593 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.153571    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "kube-api-access-4hct8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:46:09.153918 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.153882    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:46:09.153998 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.153964    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:46:09.158762 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.158737    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted" pod "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" (UID: "6fcbfeec-799d-4118-a2cd-2fe79a2a01a5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:46:09.252397 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.252372    4849 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-image-registry-private-configuration\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.252397 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.252394    4849 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-trusted-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.252535 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.252404    4849 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-ca-trust-extracted\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.252535 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.252413    4849 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-installation-pull-secrets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.252535 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.252424    4849 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-bound-sa-token\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.252535 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.252433    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hct8\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-kube-api-access-4hct8\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.252535 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.252441    4849 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5-registry-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:46:09.431185 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:46:09.431156    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:46:09.463293 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.463272    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-794b85596b-gvxrs"]
Oct 30 10:46:09.477334 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:09.477307    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-794b85596b-gvxrs"]
Oct 30 10:46:10.752365 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:10.752332    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" path="/var/lib/kubelet/pods/6fcbfeec-799d-4118-a2cd-2fe79a2a01a5/volumes"
Oct 30 10:46:14.135824 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:14.135795    4849 generic.go:356] "Generic (PLEG): container finished" podID="9e199587-19b7-4a4b-bdd0-761b7a6e475f" containerID="ba0045341ed615092548f2397eecf7cad2146add9835e463622087f0b8bba617" exitCode=0
Oct 30 10:46:14.136202 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:14.135834    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-v69fn" event={"ID":"9e199587-19b7-4a4b-bdd0-761b7a6e475f","Type":"ContainerDied","Data":"ba0045341ed615092548f2397eecf7cad2146add9835e463622087f0b8bba617"}
Oct 30 10:46:14.136202 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:14.135881    4849 scope.go:122] "RemoveContainer" containerID="c72186ae410aa7a8ddf608b6ce5fa204dad865cf52b9b90bd9ac91862be54c05"
Oct 30 10:46:14.136202 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:14.136159    4849 scope.go:122] "RemoveContainer" containerID="ba0045341ed615092548f2397eecf7cad2146add9835e463622087f0b8bba617"
Oct 30 10:46:14.136429 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:46:14.136355    4849 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"insights-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=insights-operator pod=insights-operator-7c4986d449-v69fn_openshift-insights(9e199587-19b7-4a4b-bdd0-761b7a6e475f)\"" pod="openshift-insights/insights-operator-7c4986d449-v69fn" podUID="9e199587-19b7-4a4b-bdd0-761b7a6e475f"
Oct 30 10:46:23.304351 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.304315    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-696d67977-zgswj"]
Oct 30 10:46:23.304855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.304722    4849 cpu_manager.go:414] "containerMap: removing container" podUID="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" containerName="registry"
Oct 30 10:46:23.304855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.304741    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" containerName="registry"
Oct 30 10:46:23.304855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.304763    4849 cpu_manager.go:414] "containerMap: removing container" podUID="9fb55617-3df2-401d-876b-2fc95f902c53" containerName="registry"
Oct 30 10:46:23.304855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.304770    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb55617-3df2-401d-876b-2fc95f902c53" containerName="registry"
Oct 30 10:46:23.304855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.304842    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fcbfeec-799d-4118-a2cd-2fe79a2a01a5" containerName="registry"
Oct 30 10:46:23.304855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.304854    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb55617-3df2-401d-876b-2fc95f902c53" containerName="registry"
Oct 30 10:46:23.309465 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.309441    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.326261 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.326232    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 10:46:23.339889 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.339853    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 10:46:23.339982 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.339937    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5f78h\""
Oct 30 10:46:23.340058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.339976    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 10:46:23.340058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.339988    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 10:46:23.341629 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.341607    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-696d67977-zgswj"]
Oct 30 10:46:23.405900 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.405879    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 10:46:23.420214 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.420194    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 10:46:23.453793 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.453769    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.453923 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.453801    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-telemeter-trusted-ca-bundle\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.453923 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.453819    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-secret-telemeter-client\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.453923 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.453836    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-metrics-client-ca\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.454106 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.453938    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-federate-client-tls\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.454144 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.454110    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhm5d\" (UniqueName: \"kubernetes.io/projected/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-kube-api-access-xhm5d\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.454182 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.454148    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-serving-certs-ca-bundle\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.454182 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.454173    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-telemeter-client-tls\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.554874 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.554817    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-secret-telemeter-client\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.554874 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.554846    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-metrics-client-ca\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555065 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555035    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-federate-client-tls\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555136 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555083    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhm5d\" (UniqueName: \"kubernetes.io/projected/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-kube-api-access-xhm5d\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555136 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555110    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-serving-certs-ca-bundle\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555136 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555128    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-telemeter-client-tls\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555282 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555152    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555282 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555173    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-telemeter-trusted-ca-bundle\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555709 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555662    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-metrics-client-ca\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.555874 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.555852    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-serving-certs-ca-bundle\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.556165 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.556142    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-telemeter-trusted-ca-bundle\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.557527 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.557505    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-federate-client-tls\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.557617 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.557552    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-telemeter-client-tls\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.557717 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.557639    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.557751 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.557718    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-secret-telemeter-client\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.579152 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.579122    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhm5d\" (UniqueName: \"kubernetes.io/projected/6c28f4af-cc57-4c19-baf3-eaeaa07e7853-kube-api-access-xhm5d\") pod \"telemeter-client-696d67977-zgswj\" (UID: \"6c28f4af-cc57-4c19-baf3-eaeaa07e7853\") " pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.627823 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.627803    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-696d67977-zgswj"
Oct 30 10:46:23.964457 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:23.964430    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-696d67977-zgswj"]
Oct 30 10:46:23.967553 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:46:23.967526    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c28f4af_cc57_4c19_baf3_eaeaa07e7853.slice/crio-0e99ff5e636c106e87cbf08f671797ad8a8459772b5b2c866f6c6e1a3d36e850 WatchSource:0}: Error finding container 0e99ff5e636c106e87cbf08f671797ad8a8459772b5b2c866f6c6e1a3d36e850: Status 404 returned error can't find the container with id 0e99ff5e636c106e87cbf08f671797ad8a8459772b5b2c866f6c6e1a3d36e850
Oct 30 10:46:24.173357 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:24.173328    4849 generic.go:356] "Generic (PLEG): container finished" podID="04a41ddb-fcb6-49bf-8468-deccd08a460c" containerID="8656aa12033f59b56f04ce7d16a7cf56e64cd119eab5c6223ddad65ad0da292c" exitCode=0
Oct 30 10:46:24.173451 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:24.173405    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6" event={"ID":"04a41ddb-fcb6-49bf-8468-deccd08a460c","Type":"ContainerDied","Data":"8656aa12033f59b56f04ce7d16a7cf56e64cd119eab5c6223ddad65ad0da292c"}
Oct 30 10:46:24.173754 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:24.173730    4849 scope.go:122] "RemoveContainer" containerID="8656aa12033f59b56f04ce7d16a7cf56e64cd119eab5c6223ddad65ad0da292c"
Oct 30 10:46:24.174827 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:24.174799    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696d67977-zgswj" event={"ID":"6c28f4af-cc57-4c19-baf3-eaeaa07e7853","Type":"ContainerStarted","Data":"0e99ff5e636c106e87cbf08f671797ad8a8459772b5b2c866f6c6e1a3d36e850"}
Oct 30 10:46:25.181591 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:25.181543    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5g7j6" event={"ID":"04a41ddb-fcb6-49bf-8468-deccd08a460c","Type":"ContainerStarted","Data":"d9142f032ca514ce54f562c96506300e6cf200effb95291ec78baa373adab012"}
Oct 30 10:46:25.747880 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:25.747809    4849 scope.go:122] "RemoveContainer" containerID="ba0045341ed615092548f2397eecf7cad2146add9835e463622087f0b8bba617"
Oct 30 10:46:26.187550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:26.187511    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-v69fn" event={"ID":"9e199587-19b7-4a4b-bdd0-761b7a6e475f","Type":"ContainerStarted","Data":"19b6d13eef663fe8b1fcd580ea54877f87ac897f5ba3ccb9fa9013f33638e61f"}
Oct 30 10:46:26.189165 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:26.189141    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696d67977-zgswj" event={"ID":"6c28f4af-cc57-4c19-baf3-eaeaa07e7853","Type":"ContainerStarted","Data":"1b5a188da5eb98513f30ad553f8d7349f9b57c4484b9a88fbde85adb8d00b20e"}
Oct 30 10:46:27.195546 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:27.195504    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696d67977-zgswj" event={"ID":"6c28f4af-cc57-4c19-baf3-eaeaa07e7853","Type":"ContainerStarted","Data":"6b7a543af4d8e9f8cbe0f6d55bd914648cf4878e2ff9842e99a12347527a2c93"}
Oct 30 10:46:27.195993 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:27.195572    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696d67977-zgswj" event={"ID":"6c28f4af-cc57-4c19-baf3-eaeaa07e7853","Type":"ContainerStarted","Data":"64b68d7181c535e19de61c94e4543cfba4a85956c44deb06bc13f83efac5e1d9"}
Oct 30 10:46:27.272661 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:27.272604    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-696d67977-zgswj" podStartSLOduration=1.9364057730000002 podStartE2EDuration="4.272591102s" totalImagesPullingTime="2.336185329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:46:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:46:27.258410961 +0000 UTC m=+84.638860402" watchObservedRunningTime="2026-10-30 10:46:27.272591102 +0000 UTC m=+84.653040625"
Oct 30 10:46:29.206102 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:29.206067    4849 generic.go:356] "Generic (PLEG): container finished" podID="7333057e-e1db-4970-bf86-eede9e94c06a" containerID="e9bd4aa37b4e865374d425a95b2a451b0b61a01d995b5c0712722a05daeed3d0" exitCode=0
Oct 30 10:46:29.206603 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:29.206150    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst" event={"ID":"7333057e-e1db-4970-bf86-eede9e94c06a","Type":"ContainerDied","Data":"e9bd4aa37b4e865374d425a95b2a451b0b61a01d995b5c0712722a05daeed3d0"}
Oct 30 10:46:29.206603 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:29.206545    4849 scope.go:122] "RemoveContainer" containerID="e9bd4aa37b4e865374d425a95b2a451b0b61a01d995b5c0712722a05daeed3d0"
Oct 30 10:46:30.213131 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:46:30.213085    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q8lst" event={"ID":"7333057e-e1db-4970-bf86-eede9e94c06a","Type":"ContainerStarted","Data":"95067111f73b503f10bd5f9ca6a4fac748642112b4c21fe7424639d799e0a7d6"}
Oct 30 10:46:39.431777 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:46:39.431737    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:47:09.431606 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:47:09.431569    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:47:39.432313 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:47:39.432229    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:48:09.432335 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:48:09.432295    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:48:39.432731 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:48:39.432691    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:49:09.431465 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:49:09.431422    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:49:39.432920 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:49:39.432873    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:50:02.758318 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:02.758289    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 10:50:02.759763 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:02.759740    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 10:50:09.430602 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:50:09.430566    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:50:10.336860 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.336828    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"]
Oct 30 10:50:10.340226 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.340206    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.356432 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.356408    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 10:50:10.369858 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.369834    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 10:50:10.369971 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.369952    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-vg772\""
Oct 30 10:50:10.371863 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.371844    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"]
Oct 30 10:50:10.382051 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.382010    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.382215 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.382126    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q96tl\" (UniqueName: \"kubernetes.io/projected/01420d5f-1882-471c-ae8e-3dc563226dc8-kube-api-access-q96tl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.382293 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.382211    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.483137 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.483101    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q96tl\" (UniqueName: \"kubernetes.io/projected/01420d5f-1882-471c-ae8e-3dc563226dc8-kube-api-access-q96tl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.483505 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.483166    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.483505 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.483225    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.483625 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.483521    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.483625 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.483583    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.507900 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.507872    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q96tl\" (UniqueName: \"kubernetes.io/projected/01420d5f-1882-471c-ae8e-3dc563226dc8-kube-api-access-q96tl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.655050 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.654990    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:10.951541 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.951481    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"]
Oct 30 10:50:10.955834 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:50:10.955805    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01420d5f_1882_471c_ae8e_3dc563226dc8.slice/crio-ec96bd193595cc4e821e72210972c05dd7948e1d860f46f058b520a006dda503 WatchSource:0}: Error finding container ec96bd193595cc4e821e72210972c05dd7948e1d860f46f058b520a006dda503: Status 404 returned error can't find the container with id ec96bd193595cc4e821e72210972c05dd7948e1d860f46f058b520a006dda503
Oct 30 10:50:10.957735 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:10.957716    4849 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 10:50:11.041758 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:11.041737    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv" event={"ID":"01420d5f-1882-471c-ae8e-3dc563226dc8","Type":"ContainerStarted","Data":"ec96bd193595cc4e821e72210972c05dd7948e1d860f46f058b520a006dda503"}
Oct 30 10:50:17.066205 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:17.066167    4849 generic.go:356] "Generic (PLEG): container finished" podID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerID="d0d9e992e76d0db4af482457865335c346ebe74bb3c15f3e055814a24db6964f" exitCode=0
Oct 30 10:50:17.066651 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:17.066244    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv" event={"ID":"01420d5f-1882-471c-ae8e-3dc563226dc8","Type":"ContainerDied","Data":"d0d9e992e76d0db4af482457865335c346ebe74bb3c15f3e055814a24db6964f"}
Oct 30 10:50:19.074876 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:19.074846    4849 generic.go:356] "Generic (PLEG): container finished" podID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerID="ce0564428eba7b39c476a089e9d0dd2bb1e5323acc55ce119c17bc12cb49ecdc" exitCode=0
Oct 30 10:50:19.075211 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:19.074923    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv" event={"ID":"01420d5f-1882-471c-ae8e-3dc563226dc8","Type":"ContainerDied","Data":"ce0564428eba7b39c476a089e9d0dd2bb1e5323acc55ce119c17bc12cb49ecdc"}
Oct 30 10:50:25.101148 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:25.101115    4849 generic.go:356] "Generic (PLEG): container finished" podID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerID="4fc7e9a0635636929d8befd205651b0d7d7cf16b32606ab86cef971582c1a3f0" exitCode=0
Oct 30 10:50:25.101503 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:25.101203    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv" event={"ID":"01420d5f-1882-471c-ae8e-3dc563226dc8","Type":"ContainerDied","Data":"4fc7e9a0635636929d8befd205651b0d7d7cf16b32606ab86cef971582c1a3f0"}
Oct 30 10:50:26.230529 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.230504    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:26.303404 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.303384    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-util\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-util\") pod \"01420d5f-1882-471c-ae8e-3dc563226dc8\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") "
Oct 30 10:50:26.303510 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.303422    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01420d5f-1882-471c-ae8e-3dc563226dc8-kube-api-access-q96tl\" (UniqueName: \"kubernetes.io/projected/01420d5f-1882-471c-ae8e-3dc563226dc8-kube-api-access-q96tl\") pod \"01420d5f-1882-471c-ae8e-3dc563226dc8\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") "
Oct 30 10:50:26.303510 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.303503    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-bundle\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-bundle\") pod \"01420d5f-1882-471c-ae8e-3dc563226dc8\" (UID: \"01420d5f-1882-471c-ae8e-3dc563226dc8\") "
Oct 30 10:50:26.304169 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.304144    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-bundle" pod "01420d5f-1882-471c-ae8e-3dc563226dc8" (UID: "01420d5f-1882-471c-ae8e-3dc563226dc8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:50:26.305637 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.305610    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01420d5f-1882-471c-ae8e-3dc563226dc8-kube-api-access-q96tl" pod "01420d5f-1882-471c-ae8e-3dc563226dc8" (UID: "01420d5f-1882-471c-ae8e-3dc563226dc8"). InnerVolumeSpecName "kube-api-access-q96tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:50:26.307322 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.307304    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-util" pod "01420d5f-1882-471c-ae8e-3dc563226dc8" (UID: "01420d5f-1882-471c-ae8e-3dc563226dc8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:50:26.404242 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.404186    4849 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:50:26.404242 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.404209    4849 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/01420d5f-1882-471c-ae8e-3dc563226dc8-util\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:50:26.404242 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:26.404223    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q96tl\" (UniqueName: \"kubernetes.io/projected/01420d5f-1882-471c-ae8e-3dc563226dc8-kube-api-access-q96tl\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:50:27.111802 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:27.111764    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv" event={"ID":"01420d5f-1882-471c-ae8e-3dc563226dc8","Type":"ContainerDied","Data":"ec96bd193595cc4e821e72210972c05dd7948e1d860f46f058b520a006dda503"}
Oct 30 10:50:27.111930 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:27.111827    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec96bd193595cc4e821e72210972c05dd7948e1d860f46f058b520a006dda503"
Oct 30 10:50:27.111930 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:27.111783    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cr62pv"
Oct 30 10:50:33.674799 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.674763    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"]
Oct 30 10:50:33.675253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.675080    4849 cpu_manager.go:414] "containerMap: removing container" podUID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerName="extract"
Oct 30 10:50:33.675253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.675093    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerName="extract"
Oct 30 10:50:33.675253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.675124    4849 cpu_manager.go:414] "containerMap: removing container" podUID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerName="pull"
Oct 30 10:50:33.675253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.675130    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerName="pull"
Oct 30 10:50:33.675253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.675138    4849 cpu_manager.go:414] "containerMap: removing container" podUID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerName="util"
Oct 30 10:50:33.675253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.675144    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerName="util"
Oct 30 10:50:33.675253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.675192    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="01420d5f-1882-471c-ae8e-3dc563226dc8" containerName="extract"
Oct 30 10:50:33.717603 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.717576    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"]
Oct 30 10:50:33.717746 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.717663    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:33.734433 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.734413    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jtcn2\""
Oct 30 10:50:33.734690 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.734673    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:50:33.734748 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.734696    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 10:50:33.852159 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.852132    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c364c1f-1f6e-452d-83c9-84259cf0c1a9-tmp\") pod \"cert-manager-operator-controller-manager-65fc875df4-f5b96\" (UID: \"2c364c1f-1f6e-452d-83c9-84259cf0c1a9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:33.852269 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.852175    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d8tx\" (UniqueName: \"kubernetes.io/projected/2c364c1f-1f6e-452d-83c9-84259cf0c1a9-kube-api-access-6d8tx\") pod \"cert-manager-operator-controller-manager-65fc875df4-f5b96\" (UID: \"2c364c1f-1f6e-452d-83c9-84259cf0c1a9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:33.953096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.953047    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c364c1f-1f6e-452d-83c9-84259cf0c1a9-tmp\") pod \"cert-manager-operator-controller-manager-65fc875df4-f5b96\" (UID: \"2c364c1f-1f6e-452d-83c9-84259cf0c1a9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:33.953096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.953085    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d8tx\" (UniqueName: \"kubernetes.io/projected/2c364c1f-1f6e-452d-83c9-84259cf0c1a9-kube-api-access-6d8tx\") pod \"cert-manager-operator-controller-manager-65fc875df4-f5b96\" (UID: \"2c364c1f-1f6e-452d-83c9-84259cf0c1a9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:33.953451 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.953431    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c364c1f-1f6e-452d-83c9-84259cf0c1a9-tmp\") pod \"cert-manager-operator-controller-manager-65fc875df4-f5b96\" (UID: \"2c364c1f-1f6e-452d-83c9-84259cf0c1a9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:33.977749 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:33.977723    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d8tx\" (UniqueName: \"kubernetes.io/projected/2c364c1f-1f6e-452d-83c9-84259cf0c1a9-kube-api-access-6d8tx\") pod \"cert-manager-operator-controller-manager-65fc875df4-f5b96\" (UID: \"2c364c1f-1f6e-452d-83c9-84259cf0c1a9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:34.039872 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:34.039852    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"
Oct 30 10:50:34.388803 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:34.388780    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96"]
Oct 30 10:50:34.391689 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:50:34.391664    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c364c1f_1f6e_452d_83c9_84259cf0c1a9.slice/crio-15167cc02d71f9aeb2b3e34e5f888030f5bd307c76155bf654d04e268bebc736 WatchSource:0}: Error finding container 15167cc02d71f9aeb2b3e34e5f888030f5bd307c76155bf654d04e268bebc736: Status 404 returned error can't find the container with id 15167cc02d71f9aeb2b3e34e5f888030f5bd307c76155bf654d04e268bebc736
Oct 30 10:50:35.144801 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:35.144765    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96" event={"ID":"2c364c1f-1f6e-452d-83c9-84259cf0c1a9","Type":"ContainerStarted","Data":"15167cc02d71f9aeb2b3e34e5f888030f5bd307c76155bf654d04e268bebc736"}
Oct 30 10:50:39.167498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:39.167443    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96" event={"ID":"2c364c1f-1f6e-452d-83c9-84259cf0c1a9","Type":"ContainerStarted","Data":"f6d12d895f10c10b717433eeb04271694216136e4316463693140a8cd06d1991"}
Oct 30 10:50:39.243336 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:39.243279    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-65fc875df4-f5b96" podStartSLOduration=2.029070495 podStartE2EDuration="6.243264543s" totalImagesPullingTime="4.214194048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:50:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:50:39.228705895 +0000 UTC m=+336.609155358" watchObservedRunningTime="2026-10-30 10:50:39.243264543 +0000 UTC m=+336.623713978"
Oct 30 10:50:39.431247 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:50:39.431178    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:50:40.321112 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.321077    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"]
Oct 30 10:50:40.323272 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.323250    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.335093 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.335073    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"]
Oct 30 10:50:40.339435 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.339407    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 10:50:40.339577 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.339516    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-h86pj\""
Oct 30 10:50:40.339577 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.339532    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 10:50:40.398983 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.398959    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a2fb647-f1e0-41a3-9f8c-8a75ad61f053-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-vtbg6\" (UID: \"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.399094 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.398987    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75b5s\" (UniqueName: \"kubernetes.io/projected/3a2fb647-f1e0-41a3-9f8c-8a75ad61f053-kube-api-access-75b5s\") pod \"cert-manager-webhook-5dbbbf75cc-vtbg6\" (UID: \"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.499711 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.499683    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a2fb647-f1e0-41a3-9f8c-8a75ad61f053-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-vtbg6\" (UID: \"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.499814 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.499716    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75b5s\" (UniqueName: \"kubernetes.io/projected/3a2fb647-f1e0-41a3-9f8c-8a75ad61f053-kube-api-access-75b5s\") pod \"cert-manager-webhook-5dbbbf75cc-vtbg6\" (UID: \"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.543177 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.543148    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a2fb647-f1e0-41a3-9f8c-8a75ad61f053-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-vtbg6\" (UID: \"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.543387 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.543362    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75b5s\" (UniqueName: \"kubernetes.io/projected/3a2fb647-f1e0-41a3-9f8c-8a75ad61f053-kube-api-access-75b5s\") pod \"cert-manager-webhook-5dbbbf75cc-vtbg6\" (UID: \"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.646885 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.646824    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:40.942555 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:40.942493    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"]
Oct 30 10:50:40.945622 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:50:40.945591    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a2fb647_f1e0_41a3_9f8c_8a75ad61f053.slice/crio-d0ff2086a3ec24784801308dbe5672276bf85188bf5720e763fc601bbaf4cd55 WatchSource:0}: Error finding container d0ff2086a3ec24784801308dbe5672276bf85188bf5720e763fc601bbaf4cd55: Status 404 returned error can't find the container with id d0ff2086a3ec24784801308dbe5672276bf85188bf5720e763fc601bbaf4cd55
Oct 30 10:50:41.176811 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:41.176782    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6" event={"ID":"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053","Type":"ContainerStarted","Data":"d0ff2086a3ec24784801308dbe5672276bf85188bf5720e763fc601bbaf4cd55"}
Oct 30 10:50:50.214603 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:50.214564    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6" event={"ID":"3a2fb647-f1e0-41a3-9f8c-8a75ad61f053","Type":"ContainerStarted","Data":"05dd91f4d13a8eca5584b6d48d13b8e8e2c5ba5fd722eb6611886106c8375c16"}
Oct 30 10:50:50.214974 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:50.214683    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:50:50.258250 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:50.258205    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6" podStartSLOduration=1.660925426 podStartE2EDuration="10.258191482s" totalImagesPullingTime="8.597266056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:50:50.256936162 +0000 UTC m=+347.637385603" watchObservedRunningTime="2026-10-30 10:50:50.258191482 +0000 UTC m=+347.638640921"
Oct 30 10:50:52.258437 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.258402    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-kvz5f"]
Oct 30 10:50:52.261753 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.261733    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.282989 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.282964    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-kvz5f"]
Oct 30 10:50:52.295361 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.295338    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wvvbh\""
Oct 30 10:50:52.378068 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.378041    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64rhh\" (UniqueName: \"kubernetes.io/projected/e1abb1f2-0199-4f07-ad7b-8779b9e93dbb-kube-api-access-64rhh\") pod \"cert-manager-597b95cb84-kvz5f\" (UID: \"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb\") " pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.378162 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.378097    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1abb1f2-0199-4f07-ad7b-8779b9e93dbb-bound-sa-token\") pod \"cert-manager-597b95cb84-kvz5f\" (UID: \"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb\") " pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.478683 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.478655    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1abb1f2-0199-4f07-ad7b-8779b9e93dbb-bound-sa-token\") pod \"cert-manager-597b95cb84-kvz5f\" (UID: \"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb\") " pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.478796 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.478707    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64rhh\" (UniqueName: \"kubernetes.io/projected/e1abb1f2-0199-4f07-ad7b-8779b9e93dbb-kube-api-access-64rhh\") pod \"cert-manager-597b95cb84-kvz5f\" (UID: \"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb\") " pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.503136 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.503107    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1abb1f2-0199-4f07-ad7b-8779b9e93dbb-bound-sa-token\") pod \"cert-manager-597b95cb84-kvz5f\" (UID: \"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb\") " pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.503262 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.503168    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64rhh\" (UniqueName: \"kubernetes.io/projected/e1abb1f2-0199-4f07-ad7b-8779b9e93dbb-kube-api-access-64rhh\") pod \"cert-manager-597b95cb84-kvz5f\" (UID: \"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb\") " pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.572804 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.572780    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-kvz5f"
Oct 30 10:50:52.868463 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:52.868398    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-kvz5f"]
Oct 30 10:50:52.871675 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:50:52.871642    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1abb1f2_0199_4f07_ad7b_8779b9e93dbb.slice/crio-913d7bb7ad60015d4dd2e3a8c8c4b2537e5aa9755ef79b24825b172ec65e2dac WatchSource:0}: Error finding container 913d7bb7ad60015d4dd2e3a8c8c4b2537e5aa9755ef79b24825b172ec65e2dac: Status 404 returned error can't find the container with id 913d7bb7ad60015d4dd2e3a8c8c4b2537e5aa9755ef79b24825b172ec65e2dac
Oct 30 10:50:53.229998 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:53.229922    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-kvz5f" event={"ID":"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb","Type":"ContainerStarted","Data":"89a5be3d90b30bba5cecf74dad74c29a35079a00c1af1d4c97d13ddc43287476"}
Oct 30 10:50:53.229998 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:53.229969    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-kvz5f" event={"ID":"e1abb1f2-0199-4f07-ad7b-8779b9e93dbb","Type":"ContainerStarted","Data":"913d7bb7ad60015d4dd2e3a8c8c4b2537e5aa9755ef79b24825b172ec65e2dac"}
Oct 30 10:50:53.273227 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:53.273057    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-kvz5f" podStartSLOduration=1.273040347 podStartE2EDuration="1.273040347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:50:53.272425511 +0000 UTC m=+350.652874950" watchObservedRunningTime="2026-10-30 10:50:53.273040347 +0000 UTC m=+350.653489779"
Oct 30 10:50:56.221974 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:50:56.221933    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-vtbg6"
Oct 30 10:51:09.433270 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:51:09.433223    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:51:39.429764 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:51:39.429731    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:52:09.432242 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:52:09.432195    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:52:39.431692 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:52:39.431656    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:52:52.005510 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.005470    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"]
Oct 30 10:52:52.009013 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.008988    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.041433 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.041408    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 10:52:52.041562 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.041521    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 10:52:52.041562 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.041527    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"07f0b9f07fd02dc1509b226b57404b9c198c3c-8adea440-dockercfg-4c4h5\""
Oct 30 10:52:52.042872 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.042851    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"]
Oct 30 10:52:52.094233 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.094209    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tjh5\" (UniqueName: \"kubernetes.io/projected/41bcf8ae-c4fd-4165-96e8-e65c9801b723-kube-api-access-2tjh5\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.094336 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.094261    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-bundle\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.094407 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.094380    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-util\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.101475 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.101448    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"]
Oct 30 10:52:52.105081 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.105063    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.125123 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.125103    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"]
Oct 30 10:52:52.128882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.128859    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e1df189bb31fb8c419c1e848482afda4dccb51-08dd003f-dockercfg-rxg5c\""
Oct 30 10:52:52.195640 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.195613    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tjh5\" (UniqueName: \"kubernetes.io/projected/41bcf8ae-c4fd-4165-96e8-e65c9801b723-kube-api-access-2tjh5\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.195745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.195665    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-bundle\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.195745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.195702    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmgkb\" (UniqueName: \"kubernetes.io/projected/26e946d9-4bd6-49ed-8894-bece1e53954f-kube-api-access-vmgkb\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.195745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.195737    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-util\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.195860 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.195801    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-bundle\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.195860 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.195836    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-util\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.196181 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.196163    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-util\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.196227 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.196190    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-bundle\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.221285 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.221264    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tjh5\" (UniqueName: \"kubernetes.io/projected/41bcf8ae-c4fd-4165-96e8-e65c9801b723-kube-api-access-2tjh5\") pod \"07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") " pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.296904 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.296851    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-bundle\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.296904 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.296881    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-util\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.297053 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.296952    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmgkb\" (UniqueName: \"kubernetes.io/projected/26e946d9-4bd6-49ed-8894-bece1e53954f-kube-api-access-vmgkb\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.297291 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.297272    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-bundle\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.297291 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.297284    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-util\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.321158 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.321133    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmgkb\" (UniqueName: \"kubernetes.io/projected/26e946d9-4bd6-49ed-8894-bece1e53954f-kube-api-access-vmgkb\") pod \"e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") " pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.325776 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.325750    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:52.421615 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.421584    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:52.622390 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.622360    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"]
Oct 30 10:52:52.624740 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:52:52.624713    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41bcf8ae_c4fd_4165_96e8_e65c9801b723.slice/crio-579f91e7b29cce0eec17593c231258380da352b5c83a2a6a49828aa55ed4d88e WatchSource:0}: Error finding container 579f91e7b29cce0eec17593c231258380da352b5c83a2a6a49828aa55ed4d88e: Status 404 returned error can't find the container with id 579f91e7b29cce0eec17593c231258380da352b5c83a2a6a49828aa55ed4d88e
Oct 30 10:52:52.719184 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.719154    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"]
Oct 30 10:52:52.721967 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:52:52.721927    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e946d9_4bd6_49ed_8894_bece1e53954f.slice/crio-800d7a9e91c54b36085d70d46de07625f291606eb19635d995acb2caf37e719c WatchSource:0}: Error finding container 800d7a9e91c54b36085d70d46de07625f291606eb19635d995acb2caf37e719c: Status 404 returned error can't find the container with id 800d7a9e91c54b36085d70d46de07625f291606eb19635d995acb2caf37e719c
Oct 30 10:52:52.751062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.751040    4849 generic.go:356] "Generic (PLEG): container finished" podID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerID="23ab51b57908b4f38c39109c6c83bdd1f1667d6f2b549c5e26601e27b3103cb9" exitCode=0
Oct 30 10:52:52.753382 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.753357    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4" event={"ID":"26e946d9-4bd6-49ed-8894-bece1e53954f","Type":"ContainerStarted","Data":"800d7a9e91c54b36085d70d46de07625f291606eb19635d995acb2caf37e719c"}
Oct 30 10:52:52.753465 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.753397    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b" event={"ID":"41bcf8ae-c4fd-4165-96e8-e65c9801b723","Type":"ContainerDied","Data":"23ab51b57908b4f38c39109c6c83bdd1f1667d6f2b549c5e26601e27b3103cb9"}
Oct 30 10:52:52.753465 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:52.753423    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b" event={"ID":"41bcf8ae-c4fd-4165-96e8-e65c9801b723","Type":"ContainerStarted","Data":"579f91e7b29cce0eec17593c231258380da352b5c83a2a6a49828aa55ed4d88e"}
Oct 30 10:52:53.757291 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:53.757222    4849 generic.go:356] "Generic (PLEG): container finished" podID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerID="81c39ba6e9cf6348426ea39d9ed5c2f7809b67d160561d6c0f522eb6fb415001" exitCode=0
Oct 30 10:52:53.757677 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:53.757314    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4" event={"ID":"26e946d9-4bd6-49ed-8894-bece1e53954f","Type":"ContainerDied","Data":"81c39ba6e9cf6348426ea39d9ed5c2f7809b67d160561d6c0f522eb6fb415001"}
Oct 30 10:52:53.759226 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:53.759211    4849 generic.go:356] "Generic (PLEG): container finished" podID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerID="585ea194497c54fe430c1b6f46636f90deaf49f54e27c49e215e24dd180eba62" exitCode=0
Oct 30 10:52:53.759281 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:53.759257    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b" event={"ID":"41bcf8ae-c4fd-4165-96e8-e65c9801b723","Type":"ContainerDied","Data":"585ea194497c54fe430c1b6f46636f90deaf49f54e27c49e215e24dd180eba62"}
Oct 30 10:52:54.226394 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.226365    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"]
Oct 30 10:52:54.229700 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.229685    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.248948 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.248797    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6fb1a13d619f6ee73e221b33f4898cc831d48b-0ad9862f-dockercfg-plnl7\""
Oct 30 10:52:54.250651 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.250631    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"]
Oct 30 10:52:54.312467 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.312444    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-util\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.312559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.312494    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-bundle\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.312597 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.312556    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbvgg\" (UniqueName: \"kubernetes.io/projected/8821a12a-7f57-42b2-b83f-f8386cd363d4-kube-api-access-sbvgg\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.413298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.413271    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbvgg\" (UniqueName: \"kubernetes.io/projected/8821a12a-7f57-42b2-b83f-f8386cd363d4-kube-api-access-sbvgg\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.413389 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.413324    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-util\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.413441 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.413389    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-bundle\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.413676 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.413659    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-util\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.413718 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.413698    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-bundle\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.437882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.437858    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbvgg\" (UniqueName: \"kubernetes.io/projected/8821a12a-7f57-42b2-b83f-f8386cd363d4-kube-api-access-sbvgg\") pod \"6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") " pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.545066 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.545001    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:54.765592 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.765561    4849 generic.go:356] "Generic (PLEG): container finished" podID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerID="620684597d50436d2595809bb5c21e40a7d89272a43d8249d4b076f2b08fa6db" exitCode=0
Oct 30 10:52:54.765966 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.765640    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4" event={"ID":"26e946d9-4bd6-49ed-8894-bece1e53954f","Type":"ContainerDied","Data":"620684597d50436d2595809bb5c21e40a7d89272a43d8249d4b076f2b08fa6db"}
Oct 30 10:52:54.767965 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.767938    4849 generic.go:356] "Generic (PLEG): container finished" podID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerID="a375fc62c02cf0281eff1da3d5314ccbdb1bfdc5e224a481255b4c20d7553789" exitCode=0
Oct 30 10:52:54.768105 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.767968    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b" event={"ID":"41bcf8ae-c4fd-4165-96e8-e65c9801b723","Type":"ContainerDied","Data":"a375fc62c02cf0281eff1da3d5314ccbdb1bfdc5e224a481255b4c20d7553789"}
Oct 30 10:52:54.854948 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:54.854919    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"]
Oct 30 10:52:54.858635 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:52:54.858607    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8821a12a_7f57_42b2_b83f_f8386cd363d4.slice/crio-7b0d7cfd15ccd10a9fa6d52c34c97a53b77dcf696e5fa2db3367bb4e7f256938 WatchSource:0}: Error finding container 7b0d7cfd15ccd10a9fa6d52c34c97a53b77dcf696e5fa2db3367bb4e7f256938: Status 404 returned error can't find the container with id 7b0d7cfd15ccd10a9fa6d52c34c97a53b77dcf696e5fa2db3367bb4e7f256938
Oct 30 10:52:55.775236 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:55.775200    4849 generic.go:356] "Generic (PLEG): container finished" podID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerID="49920d128d4ae6fbd937f2ef26c3254cc8832e2e4ce17e6431c4ae4805f18830" exitCode=0
Oct 30 10:52:55.775653 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:55.775292    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj" event={"ID":"8821a12a-7f57-42b2-b83f-f8386cd363d4","Type":"ContainerDied","Data":"49920d128d4ae6fbd937f2ef26c3254cc8832e2e4ce17e6431c4ae4805f18830"}
Oct 30 10:52:55.775653 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:55.775347    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj" event={"ID":"8821a12a-7f57-42b2-b83f-f8386cd363d4","Type":"ContainerStarted","Data":"7b0d7cfd15ccd10a9fa6d52c34c97a53b77dcf696e5fa2db3367bb4e7f256938"}
Oct 30 10:52:55.778564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:55.778539    4849 generic.go:356] "Generic (PLEG): container finished" podID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerID="7cecdb912cef791a34fe8f70fdc9e55e7e1ab2042464ee04af632f6e598138fd" exitCode=0
Oct 30 10:52:55.778663 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:55.778630    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4" event={"ID":"26e946d9-4bd6-49ed-8894-bece1e53954f","Type":"ContainerDied","Data":"7cecdb912cef791a34fe8f70fdc9e55e7e1ab2042464ee04af632f6e598138fd"}
Oct 30 10:52:55.905199 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:55.905176    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:56.028694 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.028641    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41bcf8ae-c4fd-4165-96e8-e65c9801b723-kube-api-access-2tjh5\" (UniqueName: \"kubernetes.io/projected/41bcf8ae-c4fd-4165-96e8-e65c9801b723-kube-api-access-2tjh5\") pod \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") "
Oct 30 10:52:56.028694 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.028689    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-bundle\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-bundle\") pod \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") "
Oct 30 10:52:56.028875 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.028708    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-util\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-util\") pod \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\" (UID: \"41bcf8ae-c4fd-4165-96e8-e65c9801b723\") "
Oct 30 10:52:56.029259 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.029230    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-bundle" pod "41bcf8ae-c4fd-4165-96e8-e65c9801b723" (UID: "41bcf8ae-c4fd-4165-96e8-e65c9801b723"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:52:56.030512 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.030490    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41bcf8ae-c4fd-4165-96e8-e65c9801b723-kube-api-access-2tjh5" pod "41bcf8ae-c4fd-4165-96e8-e65c9801b723" (UID: "41bcf8ae-c4fd-4165-96e8-e65c9801b723"). InnerVolumeSpecName "kube-api-access-2tjh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:52:56.034515 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.034482    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-util" pod "41bcf8ae-c4fd-4165-96e8-e65c9801b723" (UID: "41bcf8ae-c4fd-4165-96e8-e65c9801b723"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:52:56.130119 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.130094    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tjh5\" (UniqueName: \"kubernetes.io/projected/41bcf8ae-c4fd-4165-96e8-e65c9801b723-kube-api-access-2tjh5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:56.130119 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.130115    4849 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:56.130244 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.130126    4849 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41bcf8ae-c4fd-4165-96e8-e65c9801b723-util\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:56.783679 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.783659    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b"
Oct 30 10:52:56.783976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.783652    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/07f0b9f07fd02dc1509b226b57404b9c198c3c836737a4f34809a10054jpg7b" event={"ID":"41bcf8ae-c4fd-4165-96e8-e65c9801b723","Type":"ContainerDied","Data":"579f91e7b29cce0eec17593c231258380da352b5c83a2a6a49828aa55ed4d88e"}
Oct 30 10:52:56.783976 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.783779    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="579f91e7b29cce0eec17593c231258380da352b5c83a2a6a49828aa55ed4d88e"
Oct 30 10:52:56.785542 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.785504    4849 generic.go:356] "Generic (PLEG): container finished" podID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerID="c949f30ae657a533323c089ddb5e4afabf4db7421d1769410304673cc31bba34" exitCode=0
Oct 30 10:52:56.785645 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.785539    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj" event={"ID":"8821a12a-7f57-42b2-b83f-f8386cd363d4","Type":"ContainerDied","Data":"c949f30ae657a533323c089ddb5e4afabf4db7421d1769410304673cc31bba34"}
Oct 30 10:52:56.916916 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:56.916894    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:57.036364 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.036327    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26e946d9-4bd6-49ed-8894-bece1e53954f-kube-api-access-vmgkb\" (UniqueName: \"kubernetes.io/projected/26e946d9-4bd6-49ed-8894-bece1e53954f-kube-api-access-vmgkb\") pod \"26e946d9-4bd6-49ed-8894-bece1e53954f\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") "
Oct 30 10:52:57.036503 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.036414    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-bundle\") pod \"26e946d9-4bd6-49ed-8894-bece1e53954f\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") "
Oct 30 10:52:57.036503 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.036444    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-util\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-util\") pod \"26e946d9-4bd6-49ed-8894-bece1e53954f\" (UID: \"26e946d9-4bd6-49ed-8894-bece1e53954f\") "
Oct 30 10:52:57.037114 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.037085    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-bundle" pod "26e946d9-4bd6-49ed-8894-bece1e53954f" (UID: "26e946d9-4bd6-49ed-8894-bece1e53954f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:52:57.038681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.038652    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e946d9-4bd6-49ed-8894-bece1e53954f-kube-api-access-vmgkb" pod "26e946d9-4bd6-49ed-8894-bece1e53954f" (UID: "26e946d9-4bd6-49ed-8894-bece1e53954f"). InnerVolumeSpecName "kube-api-access-vmgkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:52:57.042638 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.042609    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-util" pod "26e946d9-4bd6-49ed-8894-bece1e53954f" (UID: "26e946d9-4bd6-49ed-8894-bece1e53954f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:52:57.137201 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.137151    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmgkb\" (UniqueName: \"kubernetes.io/projected/26e946d9-4bd6-49ed-8894-bece1e53954f-kube-api-access-vmgkb\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:57.137201 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.137173    4849 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:57.137201 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.137182    4849 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26e946d9-4bd6-49ed-8894-bece1e53954f-util\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:57.792888 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.792859    4849 generic.go:356] "Generic (PLEG): container finished" podID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerID="e1503b8f533fc5673b6db61f571c10ad35e8014265dde59c5d072cd59562daab" exitCode=0
Oct 30 10:52:57.793322 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.792957    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj" event={"ID":"8821a12a-7f57-42b2-b83f-f8386cd363d4","Type":"ContainerDied","Data":"e1503b8f533fc5673b6db61f571c10ad35e8014265dde59c5d072cd59562daab"}
Oct 30 10:52:57.795140 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.795124    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4"
Oct 30 10:52:57.795140 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.795131    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e1df189bb31fb8c419c1e848482afda4dccb516290aeca4a9ece71e9742cmx4" event={"ID":"26e946d9-4bd6-49ed-8894-bece1e53954f","Type":"ContainerDied","Data":"800d7a9e91c54b36085d70d46de07625f291606eb19635d995acb2caf37e719c"}
Oct 30 10:52:57.795246 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:57.795168    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="800d7a9e91c54b36085d70d46de07625f291606eb19635d995acb2caf37e719c"
Oct 30 10:52:58.924524 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:58.924503    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:59.052181 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.052113    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-util\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-util\") pod \"8821a12a-7f57-42b2-b83f-f8386cd363d4\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") "
Oct 30 10:52:59.052181 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.052183    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-bundle\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-bundle\") pod \"8821a12a-7f57-42b2-b83f-f8386cd363d4\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") "
Oct 30 10:52:59.052346 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.052213    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8821a12a-7f57-42b2-b83f-f8386cd363d4-kube-api-access-sbvgg\" (UniqueName: \"kubernetes.io/projected/8821a12a-7f57-42b2-b83f-f8386cd363d4-kube-api-access-sbvgg\") pod \"8821a12a-7f57-42b2-b83f-f8386cd363d4\" (UID: \"8821a12a-7f57-42b2-b83f-f8386cd363d4\") "
Oct 30 10:52:59.053643 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.053614    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-bundle" pod "8821a12a-7f57-42b2-b83f-f8386cd363d4" (UID: "8821a12a-7f57-42b2-b83f-f8386cd363d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:52:59.054198 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.054175    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8821a12a-7f57-42b2-b83f-f8386cd363d4-kube-api-access-sbvgg" pod "8821a12a-7f57-42b2-b83f-f8386cd363d4" (UID: "8821a12a-7f57-42b2-b83f-f8386cd363d4"). InnerVolumeSpecName "kube-api-access-sbvgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:52:59.057255 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.057236    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-util" pod "8821a12a-7f57-42b2-b83f-f8386cd363d4" (UID: "8821a12a-7f57-42b2-b83f-f8386cd363d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:52:59.153167 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.153146    4849 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-util\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:59.153167 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.153165    4849 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8821a12a-7f57-42b2-b83f-f8386cd363d4-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:59.153289 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.153175    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbvgg\" (UniqueName: \"kubernetes.io/projected/8821a12a-7f57-42b2-b83f-f8386cd363d4-kube-api-access-sbvgg\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:52:59.805807 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.805779    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj" event={"ID":"8821a12a-7f57-42b2-b83f-f8386cd363d4","Type":"ContainerDied","Data":"7b0d7cfd15ccd10a9fa6d52c34c97a53b77dcf696e5fa2db3367bb4e7f256938"}
Oct 30 10:52:59.805807 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.805800    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6fb1a13d619f6ee73e221b33f4898cc831d48b9998b06378838f67eccf5qkfj"
Oct 30 10:52:59.805967 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:52:59.805830    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b0d7cfd15ccd10a9fa6d52c34c97a53b77dcf696e5fa2db3367bb4e7f256938"
Oct 30 10:53:03.720978 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.720942    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"]
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721254    4849 cpu_manager.go:414] "containerMap: removing container" podUID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerName="pull"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721265    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerName="pull"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721279    4849 cpu_manager.go:414] "containerMap: removing container" podUID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerName="util"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721284    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerName="util"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721291    4849 cpu_manager.go:414] "containerMap: removing container" podUID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerName="extract"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721296    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerName="extract"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721303    4849 cpu_manager.go:414] "containerMap: removing container" podUID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerName="extract"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721308    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerName="extract"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721319    4849 cpu_manager.go:414] "containerMap: removing container" podUID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerName="pull"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721324    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerName="pull"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721333    4849 cpu_manager.go:414] "containerMap: removing container" podUID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerName="util"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721338    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerName="util"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721354    4849 cpu_manager.go:414] "containerMap: removing container" podUID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerName="util"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721360    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerName="util"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721367    4849 cpu_manager.go:414] "containerMap: removing container" podUID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerName="extract"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721374    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerName="extract"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721379    4849 cpu_manager.go:414] "containerMap: removing container" podUID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerName="pull"
Oct 30 10:53:03.721381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721383    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerName="pull"
Oct 30 10:53:03.721891 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721445    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="41bcf8ae-c4fd-4165-96e8-e65c9801b723" containerName="extract"
Oct 30 10:53:03.721891 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721452    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="8821a12a-7f57-42b2-b83f-f8386cd363d4" containerName="extract"
Oct 30 10:53:03.721891 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.721460    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="26e946d9-4bd6-49ed-8894-bece1e53954f" containerName="extract"
Oct 30 10:53:03.725547 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.725524    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.742073 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.742015    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-tempo-operator\"/\"tempo-operator-controller-service-cert\""
Oct 30 10:53:03.742073 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.742058    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-tempo-operator\"/\"webhook-server-cert\""
Oct 30 10:53:03.742259 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.742196    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-tempo-operator\"/\"kube-root-ca.crt\""
Oct 30 10:53:03.755707 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.755682    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-tempo-operator\"/\"tempo-operator-metrics\""
Oct 30 10:53:03.755830 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.755754    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-tempo-operator\"/\"tempo-operator-controller-manager-dockercfg-865ng\""
Oct 30 10:53:03.757837 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.757811    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"]
Oct 30 10:53:03.841981 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.841962    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-tempo-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:53:03.893860 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.893836    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjlnz\" (UniqueName: \"kubernetes.io/projected/53623e32-f8fc-455f-900f-fcd62ffe0518-kube-api-access-gjlnz\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.893975 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.893875    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-webhook-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.893975 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.893952    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-apiservice-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.894119 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.894104    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-tempo-operator-metrics-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.995264 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.995213    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjlnz\" (UniqueName: \"kubernetes.io/projected/53623e32-f8fc-455f-900f-fcd62ffe0518-kube-api-access-gjlnz\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.995264 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.995242    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-webhook-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.995377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.995269    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-apiservice-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.995377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.995307    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-tempo-operator-metrics-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.998001 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.997974    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-webhook-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.998106 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.997974    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-tempo-operator-metrics-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:03.998106 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:03.998048    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53623e32-f8fc-455f-900f-fcd62ffe0518-apiservice-cert\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:04.019364 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:04.019341    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjlnz\" (UniqueName: \"kubernetes.io/projected/53623e32-f8fc-455f-900f-fcd62ffe0518-kube-api-access-gjlnz\") pod \"tempo-operator-controller-645f46b65d-45v2v\" (UID: \"53623e32-f8fc-455f-900f-fcd62ffe0518\") " pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:04.037488 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:04.037468    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:04.394775 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:04.394747    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"]
Oct 30 10:53:04.397455 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:53:04.397428    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53623e32_f8fc_455f_900f_fcd62ffe0518.slice/crio-36cbbcfdaa7eba503708e48f0ff03c40b887778507c220dabd37072f74243f9b WatchSource:0}: Error finding container 36cbbcfdaa7eba503708e48f0ff03c40b887778507c220dabd37072f74243f9b: Status 404 returned error can't find the container with id 36cbbcfdaa7eba503708e48f0ff03c40b887778507c220dabd37072f74243f9b
Oct 30 10:53:04.828240 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:04.828205    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v" event={"ID":"53623e32-f8fc-455f-900f-fcd62ffe0518","Type":"ContainerStarted","Data":"36cbbcfdaa7eba503708e48f0ff03c40b887778507c220dabd37072f74243f9b"}
Oct 30 10:53:06.838763 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:06.838736    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v" event={"ID":"53623e32-f8fc-455f-900f-fcd62ffe0518","Type":"ContainerStarted","Data":"ccf8d179f7ae5c8d13e99ed252075707a4eb7b4f7e6c3b972dff75b88a05d5bd"}
Oct 30 10:53:06.839115 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:06.838842    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:06.915369 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:06.915318    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v" podStartSLOduration=1.903337039 podStartE2EDuration="3.915306114s" totalImagesPullingTime="2.011969075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:53:06.9012679 +0000 UTC m=+484.281717338" watchObservedRunningTime="2026-10-30 10:53:06.915306114 +0000 UTC m=+484.295755552"
Oct 30 10:53:09.430801 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:53:09.430763    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:53:09.941872 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:09.941843    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"]
Oct 30 10:53:09.945090 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:09.945072    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:09.962723 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:09.962703    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-opentelemetry-operator\"/\"kube-root-ca.crt\""
Oct 30 10:53:09.976153 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:09.976123    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-opentelemetry-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:53:09.976296 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:09.976282    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-opentelemetry-operator\"/\"opentelemetry-operator-controller-manager-dockercfg-tdlw7\""
Oct 30 10:53:09.976364 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:09.976351    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-opentelemetry-operator\"/\"opentelemetry-operator-metrics\""
Oct 30 10:53:09.977757 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:09.977733    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"]
Oct 30 10:53:10.140902 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.140875    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"opentelemetry-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/741b3347-5787-4d66-a0f1-2c76c969a41b-opentelemetry-operator-metrics-cert\") pod \"opentelemetry-operator-controller-manager-5f695b7fbf-zspqv\" (UID: \"741b3347-5787-4d66-a0f1-2c76c969a41b\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:10.141013 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.140930    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ghz\" (UniqueName: \"kubernetes.io/projected/741b3347-5787-4d66-a0f1-2c76c969a41b-kube-api-access-59ghz\") pod \"opentelemetry-operator-controller-manager-5f695b7fbf-zspqv\" (UID: \"741b3347-5787-4d66-a0f1-2c76c969a41b\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:10.241404 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.241329    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"opentelemetry-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/741b3347-5787-4d66-a0f1-2c76c969a41b-opentelemetry-operator-metrics-cert\") pod \"opentelemetry-operator-controller-manager-5f695b7fbf-zspqv\" (UID: \"741b3347-5787-4d66-a0f1-2c76c969a41b\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:10.241514 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.241425    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ghz\" (UniqueName: \"kubernetes.io/projected/741b3347-5787-4d66-a0f1-2c76c969a41b-kube-api-access-59ghz\") pod \"opentelemetry-operator-controller-manager-5f695b7fbf-zspqv\" (UID: \"741b3347-5787-4d66-a0f1-2c76c969a41b\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:10.244012 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.243985    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"opentelemetry-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/741b3347-5787-4d66-a0f1-2c76c969a41b-opentelemetry-operator-metrics-cert\") pod \"opentelemetry-operator-controller-manager-5f695b7fbf-zspqv\" (UID: \"741b3347-5787-4d66-a0f1-2c76c969a41b\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:10.266448 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.266424    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ghz\" (UniqueName: \"kubernetes.io/projected/741b3347-5787-4d66-a0f1-2c76c969a41b-kube-api-access-59ghz\") pod \"opentelemetry-operator-controller-manager-5f695b7fbf-zspqv\" (UID: \"741b3347-5787-4d66-a0f1-2c76c969a41b\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:10.556011 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.555962    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:10.634325 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.634299    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"]
Oct 30 10:53:10.638757 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.638735    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.680220 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.680195    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"]
Oct 30 10:53:10.740550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.740534    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-opentelemetry-operator\"/\"opentelemetry-operator-webhook-service-cert\""
Oct 30 10:53:10.743865 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.743844    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6d86\" (UniqueName: \"kubernetes.io/projected/ff7d16e0-fbc1-4a04-83c0-a7818552396a-kube-api-access-t6d86\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.743922 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.743880    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff7d16e0-fbc1-4a04-83c0-a7818552396a-apiservice-cert\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.743922 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.743907    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff7d16e0-fbc1-4a04-83c0-a7818552396a-webhook-cert\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.845213 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.845182    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6d86\" (UniqueName: \"kubernetes.io/projected/ff7d16e0-fbc1-4a04-83c0-a7818552396a-kube-api-access-t6d86\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.845338 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.845228    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff7d16e0-fbc1-4a04-83c0-a7818552396a-apiservice-cert\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.845338 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.845264    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff7d16e0-fbc1-4a04-83c0-a7818552396a-webhook-cert\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.847590 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.847570    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff7d16e0-fbc1-4a04-83c0-a7818552396a-apiservice-cert\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.847675 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.847609    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff7d16e0-fbc1-4a04-83c0-a7818552396a-webhook-cert\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.878051 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.877945    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"]
Oct 30 10:53:10.878387 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.878365    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6d86\" (UniqueName: \"kubernetes.io/projected/ff7d16e0-fbc1-4a04-83c0-a7818552396a-kube-api-access-t6d86\") pod \"opentelemetry-operator-webhook-bd4f6797b-vlzfc\" (UID: \"ff7d16e0-fbc1-4a04-83c0-a7818552396a\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:10.880729 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:53:10.880708    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741b3347_5787_4d66_a0f1_2c76c969a41b.slice/crio-b90f73883f95d791e46d3628a554c5b6b082b2bf632f6d146477779210fd9fc2 WatchSource:0}: Error finding container b90f73883f95d791e46d3628a554c5b6b082b2bf632f6d146477779210fd9fc2: Status 404 returned error can't find the container with id b90f73883f95d791e46d3628a554c5b6b082b2bf632f6d146477779210fd9fc2
Oct 30 10:53:10.950546 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:10.950523    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:11.261445 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:11.261420    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"]
Oct 30 10:53:11.263499 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:53:11.263467    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7d16e0_fbc1_4a04_83c0_a7818552396a.slice/crio-11533fafaa19a83bb4fe2ec99899e33368cf02a1bed4254484eeb9c98d00d8b4 WatchSource:0}: Error finding container 11533fafaa19a83bb4fe2ec99899e33368cf02a1bed4254484eeb9c98d00d8b4: Status 404 returned error can't find the container with id 11533fafaa19a83bb4fe2ec99899e33368cf02a1bed4254484eeb9c98d00d8b4
Oct 30 10:53:11.864241 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:11.864204    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc" event={"ID":"ff7d16e0-fbc1-4a04-83c0-a7818552396a","Type":"ContainerStarted","Data":"11533fafaa19a83bb4fe2ec99899e33368cf02a1bed4254484eeb9c98d00d8b4"}
Oct 30 10:53:11.866265 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:11.866235    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv" event={"ID":"741b3347-5787-4d66-a0f1-2c76c969a41b","Type":"ContainerStarted","Data":"b90f73883f95d791e46d3628a554c5b6b082b2bf632f6d146477779210fd9fc2"}
Oct 30 10:53:12.875376 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:12.875057    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv" event={"ID":"741b3347-5787-4d66-a0f1-2c76c969a41b","Type":"ContainerStarted","Data":"8085f8614e3d303f099b9bab6dd312ed4bae0495044dadbd51a6de02fc6edfda"}
Oct 30 10:53:12.875376 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:12.875138    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:12.940975 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:12.938410    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv" podStartSLOduration=2.050469797 podStartE2EDuration="3.93839401s" totalImagesPullingTime="1.887924213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:53:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:53:12.937445212 +0000 UTC m=+490.317894650" watchObservedRunningTime="2026-10-30 10:53:12.93839401 +0000 UTC m=+490.318843449"
Oct 30 10:53:13.883357 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:13.883317    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc" event={"ID":"ff7d16e0-fbc1-4a04-83c0-a7818552396a","Type":"ContainerStarted","Data":"e7bbaf0dc376bbb33fbb504999a50de5885cff0deae00fd49b72f8fa0444f028"}
Oct 30 10:53:13.883728 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:13.883386    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:13.992835 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:13.992769    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc" podStartSLOduration=2.293672671 podStartE2EDuration="3.992743316s" totalImagesPullingTime="1.699070645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:53:13.976930239 +0000 UTC m=+491.357379678" watchObservedRunningTime="2026-10-30 10:53:13.992743316 +0000 UTC m=+491.373192758"
Oct 30 10:53:17.849313 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:17.849229    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-tempo-operator/tempo-operator-controller-645f46b65d-45v2v"
Oct 30 10:53:21.655792 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.655755    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"]
Oct 30 10:53:21.659482 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.659461    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:21.691500 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.691475    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"]
Oct 30 10:53:21.733669 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.733644    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63280b82-d8d8-49cc-8d1b-4fdac033d073-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-748bf975df-fr95s\" (UID: \"63280b82-d8d8-49cc-8d1b-4fdac033d073\") " pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:21.733793 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.733686    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63280b82-d8d8-49cc-8d1b-4fdac033d073-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-748bf975df-fr95s\" (UID: \"63280b82-d8d8-49cc-8d1b-4fdac033d073\") " pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:21.752925 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.752899    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-observability-operator\"/\"obo-prometheus-operator-admission-webhook-service-cert\""
Oct 30 10:53:21.753048 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.753013    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-observability-operator\"/\"obo-prometheus-operator-admission-webhook-dockercfg-9hbqg\""
Oct 30 10:53:21.834498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.834469    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63280b82-d8d8-49cc-8d1b-4fdac033d073-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-748bf975df-fr95s\" (UID: \"63280b82-d8d8-49cc-8d1b-4fdac033d073\") " pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:21.834619 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.834503    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63280b82-d8d8-49cc-8d1b-4fdac033d073-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-748bf975df-fr95s\" (UID: \"63280b82-d8d8-49cc-8d1b-4fdac033d073\") " pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:21.836878 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.836855    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63280b82-d8d8-49cc-8d1b-4fdac033d073-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-748bf975df-fr95s\" (UID: \"63280b82-d8d8-49cc-8d1b-4fdac033d073\") " pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:21.836992 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.836890    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63280b82-d8d8-49cc-8d1b-4fdac033d073-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-748bf975df-fr95s\" (UID: \"63280b82-d8d8-49cc-8d1b-4fdac033d073\") " pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:21.972966 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:21.972897    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"
Oct 30 10:53:22.378758 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.378728    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s"]
Oct 30 10:53:22.381240 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:53:22.381211    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63280b82_d8d8_49cc_8d1b_4fdac033d073.slice/crio-160b2eaf30862e464079d4734250b0e032eb20f6be4a7842d941f86a6787434b WatchSource:0}: Error finding container 160b2eaf30862e464079d4734250b0e032eb20f6be4a7842d941f86a6787434b: Status 404 returned error can't find the container with id 160b2eaf30862e464079d4734250b0e032eb20f6be4a7842d941f86a6787434b
Oct 30 10:53:22.584776 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.579853    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"]
Oct 30 10:53:22.586061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.586036    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.604421 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.604397    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-observability-operator\"/\"perses-operator-service-cert\""
Oct 30 10:53:22.604552 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.604436    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-observability-operator\"/\"openshift-service-ca.crt\""
Oct 30 10:53:22.604552 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.604466    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-observability-operator\"/\"kube-root-ca.crt\""
Oct 30 10:53:22.617931 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.617911    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-observability-operator\"/\"perses-operator-dockercfg-pkt8z\""
Oct 30 10:53:22.620686 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.620659    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"]
Oct 30 10:53:22.641443 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.641392    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6aeb6327-6ca1-453b-8751-de50c92c39a3-apiservice-cert\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.641443 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.641427    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aeb6327-6ca1-453b-8751-de50c92c39a3-openshift-service-ca\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.641574 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.641447    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aeb6327-6ca1-453b-8751-de50c92c39a3-webhook-cert\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.641574 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.641461    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r8x6\" (UniqueName: \"kubernetes.io/projected/6aeb6327-6ca1-453b-8751-de50c92c39a3-kube-api-access-5r8x6\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.742793 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.742766    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aeb6327-6ca1-453b-8751-de50c92c39a3-openshift-service-ca\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.743206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.742798    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aeb6327-6ca1-453b-8751-de50c92c39a3-webhook-cert\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.743206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.742814    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r8x6\" (UniqueName: \"kubernetes.io/projected/6aeb6327-6ca1-453b-8751-de50c92c39a3-kube-api-access-5r8x6\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.743206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.742862    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6aeb6327-6ca1-453b-8751-de50c92c39a3-apiservice-cert\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.743541 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.743519    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aeb6327-6ca1-453b-8751-de50c92c39a3-openshift-service-ca\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.745169 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.745151    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6aeb6327-6ca1-453b-8751-de50c92c39a3-apiservice-cert\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.745229 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.745196    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aeb6327-6ca1-453b-8751-de50c92c39a3-webhook-cert\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.768476 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.768451    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r8x6\" (UniqueName: \"kubernetes.io/projected/6aeb6327-6ca1-453b-8751-de50c92c39a3-kube-api-access-5r8x6\") pod \"perses-operator-7bd577bd56-dzk2g\" (UID: \"6aeb6327-6ca1-453b-8751-de50c92c39a3\") " pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.898953 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.898890    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:22.925221 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:22.925195    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s" event={"ID":"63280b82-d8d8-49cc-8d1b-4fdac033d073","Type":"ContainerStarted","Data":"160b2eaf30862e464079d4734250b0e032eb20f6be4a7842d941f86a6787434b"}
Oct 30 10:53:23.236184 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:23.235691    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"]
Oct 30 10:53:23.239577 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:53:23.239546    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aeb6327_6ca1_453b_8751_de50c92c39a3.slice/crio-8fe5dcca45f36ee9e16ad180a5be49712fd2711b165085da9b3c7666dec9de00 WatchSource:0}: Error finding container 8fe5dcca45f36ee9e16ad180a5be49712fd2711b165085da9b3c7666dec9de00: Status 404 returned error can't find the container with id 8fe5dcca45f36ee9e16ad180a5be49712fd2711b165085da9b3c7666dec9de00
Oct 30 10:53:23.885226 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:23.885195    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-opentelemetry-operator/opentelemetry-operator-controller-manager-5f695b7fbf-zspqv"
Oct 30 10:53:23.932892 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:23.932857    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g" event={"ID":"6aeb6327-6ca1-453b-8751-de50c92c39a3","Type":"ContainerStarted","Data":"8fe5dcca45f36ee9e16ad180a5be49712fd2711b165085da9b3c7666dec9de00"}
Oct 30 10:53:25.895053 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:25.894925    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-bd4f6797b-vlzfc"
Oct 30 10:53:25.946098 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:25.946057    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s" event={"ID":"63280b82-d8d8-49cc-8d1b-4fdac033d073","Type":"ContainerStarted","Data":"848baf29446cf21944c7043cc2565c77d073365d41e94d5eea3e98d7de8b47bf"}
Oct 30 10:53:25.947559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:25.947530    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g" event={"ID":"6aeb6327-6ca1-453b-8751-de50c92c39a3","Type":"ContainerStarted","Data":"5eb3bd21f180116d162bc0d628465ceb3832d2aac26eeb50156e137adacabcc5"}
Oct 30 10:53:25.947684 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:25.947584    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:26.007161 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:26.007100    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-observability-operator/obo-prometheus-operator-admission-webhook-748bf975df-fr95s" podStartSLOduration=2.216845749 podStartE2EDuration="5.007082249s" totalImagesPullingTime="2.7902365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:53:26.005904042 +0000 UTC m=+503.386353482" watchObservedRunningTime="2026-10-30 10:53:26.007082249 +0000 UTC m=+503.387531689"
Oct 30 10:53:26.056102 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:26.056014    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g" podStartSLOduration=1.700881461 podStartE2EDuration="4.055995635s" totalImagesPullingTime="2.355114174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:53:26.055617373 +0000 UTC m=+503.436066811" watchObservedRunningTime="2026-10-30 10:53:26.055995635 +0000 UTC m=+503.436445077"
Oct 30 10:53:36.954307 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:36.954266    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-cluster-observability-operator/perses-operator-7bd577bd56-dzk2g"
Oct 30 10:53:39.430424 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:53:39.430390    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:53:40.541386 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.541352    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"]
Oct 30 10:53:40.548282 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.548259    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.567928 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.567906    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"]
Oct 30 10:53:40.579591 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.579569    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 10:53:40.579783 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.579767    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"d28b81ce69fab5e58d2cd73462dde4041bbc20-487d35f4-dockercfg-h5mml\""
Oct 30 10:53:40.579855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.579788    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 10:53:40.677834 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.677812    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-bundle\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.677935 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.677842    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkktk\" (UniqueName: \"kubernetes.io/projected/94a7f52b-42b7-4386-a770-b60a4332c9ea-kube-api-access-mkktk\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.677935 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.677861    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-util\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.778338 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.778309    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-bundle\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.778430 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.778346    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkktk\" (UniqueName: \"kubernetes.io/projected/94a7f52b-42b7-4386-a770-b60a4332c9ea-kube-api-access-mkktk\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.778430 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.778366    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-util\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.778764 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.778743    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-bundle\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.778824 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.778759    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-util\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.803780 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.803725    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkktk\" (UniqueName: \"kubernetes.io/projected/94a7f52b-42b7-4386-a770-b60a4332c9ea-kube-api-access-mkktk\") pod \"d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") " pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:40.865638 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:40.865613    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:41.162799 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:41.162776    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"]
Oct 30 10:53:41.165250 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:53:41.165226    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a7f52b_42b7_4386_a770_b60a4332c9ea.slice/crio-b5c7cd636a95c5aec8de3a43eece6c186dd37c64eaa2b21ce96c5f9f473f4e03 WatchSource:0}: Error finding container b5c7cd636a95c5aec8de3a43eece6c186dd37c64eaa2b21ce96c5f9f473f4e03: Status 404 returned error can't find the container with id b5c7cd636a95c5aec8de3a43eece6c186dd37c64eaa2b21ce96c5f9f473f4e03
Oct 30 10:53:42.020078 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:42.020052    4849 generic.go:356] "Generic (PLEG): container finished" podID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerID="8c5b924825b39cbdb92e376895ed3380874daf9d4572b631d44d8fe704b78d5c" exitCode=0
Oct 30 10:53:42.020440 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:42.020107    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp" event={"ID":"94a7f52b-42b7-4386-a770-b60a4332c9ea","Type":"ContainerDied","Data":"8c5b924825b39cbdb92e376895ed3380874daf9d4572b631d44d8fe704b78d5c"}
Oct 30 10:53:42.020440 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:42.020143    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp" event={"ID":"94a7f52b-42b7-4386-a770-b60a4332c9ea","Type":"ContainerStarted","Data":"b5c7cd636a95c5aec8de3a43eece6c186dd37c64eaa2b21ce96c5f9f473f4e03"}
Oct 30 10:53:44.031139 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:44.031106    4849 generic.go:356] "Generic (PLEG): container finished" podID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerID="acf5e99944d12df1b0225bca4f718d393db1c02f68cd0795965a485fded43323" exitCode=0
Oct 30 10:53:44.031475 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:44.031153    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp" event={"ID":"94a7f52b-42b7-4386-a770-b60a4332c9ea","Type":"ContainerDied","Data":"acf5e99944d12df1b0225bca4f718d393db1c02f68cd0795965a485fded43323"}
Oct 30 10:53:45.038596 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:45.038563    4849 generic.go:356] "Generic (PLEG): container finished" podID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerID="14706c6ff276dc1afd9f0b572166996dc856bff83bcfce77bf48f4df3e24997c" exitCode=0
Oct 30 10:53:45.039014 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:45.038626    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp" event={"ID":"94a7f52b-42b7-4386-a770-b60a4332c9ea","Type":"ContainerDied","Data":"14706c6ff276dc1afd9f0b572166996dc856bff83bcfce77bf48f4df3e24997c"}
Oct 30 10:53:46.175572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.175549    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:53:46.221073 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.221044    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94a7f52b-42b7-4386-a770-b60a4332c9ea-kube-api-access-mkktk\" (UniqueName: \"kubernetes.io/projected/94a7f52b-42b7-4386-a770-b60a4332c9ea-kube-api-access-mkktk\") pod \"94a7f52b-42b7-4386-a770-b60a4332c9ea\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") "
Oct 30 10:53:46.221200 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.221135    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-bundle\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-bundle\") pod \"94a7f52b-42b7-4386-a770-b60a4332c9ea\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") "
Oct 30 10:53:46.221252 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.221217    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-util\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-util\") pod \"94a7f52b-42b7-4386-a770-b60a4332c9ea\" (UID: \"94a7f52b-42b7-4386-a770-b60a4332c9ea\") "
Oct 30 10:53:46.221885 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.221841    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-bundle" pod "94a7f52b-42b7-4386-a770-b60a4332c9ea" (UID: "94a7f52b-42b7-4386-a770-b60a4332c9ea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:53:46.224046 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.223669    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a7f52b-42b7-4386-a770-b60a4332c9ea-kube-api-access-mkktk" pod "94a7f52b-42b7-4386-a770-b60a4332c9ea" (UID: "94a7f52b-42b7-4386-a770-b60a4332c9ea"). InnerVolumeSpecName "kube-api-access-mkktk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:53:46.228045 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.227998    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-util" pod "94a7f52b-42b7-4386-a770-b60a4332c9ea" (UID: "94a7f52b-42b7-4386-a770-b60a4332c9ea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:53:46.322180 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.322155    4849 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:53:46.322180 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.322178    4849 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94a7f52b-42b7-4386-a770-b60a4332c9ea-util\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:53:46.322303 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:46.322187    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkktk\" (UniqueName: \"kubernetes.io/projected/94a7f52b-42b7-4386-a770-b60a4332c9ea-kube-api-access-mkktk\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:53:47.050659 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:47.050603    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp" event={"ID":"94a7f52b-42b7-4386-a770-b60a4332c9ea","Type":"ContainerDied","Data":"b5c7cd636a95c5aec8de3a43eece6c186dd37c64eaa2b21ce96c5f9f473f4e03"}
Oct 30 10:53:47.050659 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:47.050641    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5c7cd636a95c5aec8de3a43eece6c186dd37c64eaa2b21ce96c5f9f473f4e03"
Oct 30 10:53:47.050659 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:53:47.050652    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d28b81ce69fab5e58d2cd73462dde4041bbc20368f72658a361322aaad8kfpp"
Oct 30 10:54:09.433230 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:54:09.433188    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:54:19.058491 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.058454    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:19.058991 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.058938    4849 cpu_manager.go:414] "containerMap: removing container" podUID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerName="util"
Oct 30 10:54:19.058991 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.058956    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerName="util"
Oct 30 10:54:19.058991 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.058972    4849 cpu_manager.go:414] "containerMap: removing container" podUID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerName="extract"
Oct 30 10:54:19.058991 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.058979    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerName="extract"
Oct 30 10:54:19.059299 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.059001    4849 cpu_manager.go:414] "containerMap: removing container" podUID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerName="pull"
Oct 30 10:54:19.059299 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.059009    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerName="pull"
Oct 30 10:54:19.059299 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.059124    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="94a7f52b-42b7-4386-a770-b60a4332c9ea" containerName="extract"
Oct 30 10:54:19.063449 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.063425    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.094677 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.094652    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:19.102135 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.102108    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 10:54:19.102255 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.102160    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 10:54:19.102255 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.102241    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-61f7caf9\""
Oct 30 10:54:19.106069 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.106052    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-9gvds\""
Oct 30 10:54:19.106069 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.106063    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:54:19.154066 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.154018    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6335aded-6553-4a60-af1e-1c835ccc3ba5-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.154176 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.154083    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzb9k\" (UniqueName: \"kubernetes.io/projected/6335aded-6553-4a60-af1e-1c835ccc3ba5-kube-api-access-mzb9k\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.154224 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.154174    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6335aded-6553-4a60-af1e-1c835ccc3ba5-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.255277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.255253    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzb9k\" (UniqueName: \"kubernetes.io/projected/6335aded-6553-4a60-af1e-1c835ccc3ba5-kube-api-access-mzb9k\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.255390 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.255308    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6335aded-6553-4a60-af1e-1c835ccc3ba5-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.255390 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.255384    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6335aded-6553-4a60-af1e-1c835ccc3ba5-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.256141 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.256112    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6335aded-6553-4a60-af1e-1c835ccc3ba5-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.257841 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.257815    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6335aded-6553-4a60-af1e-1c835ccc3ba5-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.279700 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.279678    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzb9k\" (UniqueName: \"kubernetes.io/projected/6335aded-6553-4a60-af1e-1c835ccc3ba5-kube-api-access-mzb9k\") pod \"data-science-collector-collector-0\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.379447 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.379389    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:19.690571 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:19.690530    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:19.693200 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:54:19.693164    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6335aded_6553_4a60_af1e_1c835ccc3ba5.slice/crio-354624cca07e1535707cc0b5c79ce74f2fe98ec99b7c0f868bfb2fa23e93ee81 WatchSource:0}: Error finding container 354624cca07e1535707cc0b5c79ce74f2fe98ec99b7c0f868bfb2fa23e93ee81: Status 404 returned error can't find the container with id 354624cca07e1535707cc0b5c79ce74f2fe98ec99b7c0f868bfb2fa23e93ee81
Oct 30 10:54:20.203656 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:20.203617    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6335aded-6553-4a60-af1e-1c835ccc3ba5","Type":"ContainerStarted","Data":"354624cca07e1535707cc0b5c79ce74f2fe98ec99b7c0f868bfb2fa23e93ee81"}
Oct 30 10:54:23.124189 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.124157    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:54:23.129014 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.128993    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.146167 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.146148    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:54:23.146259 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.146149    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-serving-cabundle\""
Oct 30 10:54:23.146259 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.146229    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-ca-bundle\""
Oct 30 10:54:23.159657 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.159607    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway-serving-cert\""
Oct 30 10:54:23.159657 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.159630    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-mtls\""
Oct 30 10:54:23.159657 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.159644    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:54:23.159896 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.159706    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-config\""
Oct 30 10:54:23.159896 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.159826    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-dockercfg-xzmfc\""
Oct 30 10:54:23.161671 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.161645    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:54:23.286788 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.286762    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.286889 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.286802    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.286889 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.286830    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.286961 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.286907    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.286961 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.286931    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vddgs\" (UniqueName: \"kubernetes.io/projected/25a38f5d-75a5-4b8b-9f22-f475046a138a-kube-api-access-vddgs\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.287047 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.286962    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.287094 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.287080    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.287147 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.287130    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387598 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387542    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vddgs\" (UniqueName: \"kubernetes.io/projected/25a38f5d-75a5-4b8b-9f22-f475046a138a-kube-api-access-vddgs\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387598 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387580    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387637    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387666    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387695    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387859    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387937 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387911    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.387993 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.387945    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.388521 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.388496    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.388626 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.388506    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.388679 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.388636    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.388735 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.388710    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.390462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.390441    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.390872 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.390853    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.390911 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.390862    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:23.411883 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:23.411864    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vddgs\" (UniqueName: \"kubernetes.io/projected/25a38f5d-75a5-4b8b-9f22-f475046a138a-kube-api-access-vddgs\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:26.915189 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:26.915118    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:26.946418 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:26.946385    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") device path: \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:27.016403 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:27.016373    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:27.016531 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:27.016515    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") DevicePath \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:27.032236 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:27.032214    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:27.216521 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:27.216490    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/fd3ed81014d76a975b5ba6f044c00684e55bfb3daddf6ff0e225ceff74c57792/globalmount\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:27.244202 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:27.244176    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:27.345723 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:27.345699    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:27.681064 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:27.680959    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:54:27.681389 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:54:27.681354    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a38f5d_75a5_4b8b_9f22_f475046a138a.slice/crio-ee65da1a02becc6ad9f6b68bfed7936661405db59ea1e767611d2d3ac4ca392e WatchSource:0}: Error finding container ee65da1a02becc6ad9f6b68bfed7936661405db59ea1e767611d2d3ac4ca392e: Status 404 returned error can't find the container with id ee65da1a02becc6ad9f6b68bfed7936661405db59ea1e767611d2d3ac4ca392e
Oct 30 10:54:28.248242 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:28.248198    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerStarted","Data":"ee65da1a02becc6ad9f6b68bfed7936661405db59ea1e767611d2d3ac4ca392e"}
Oct 30 10:54:30.262351 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:30.262255    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerStarted","Data":"7c497d2458a7ab7d218512fc08ca1206f71358b396d4b578c7f9b18a1169c767"}
Oct 30 10:54:30.665680 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:30.665648    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:54:30.665797 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:30.665721    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:54:31.270630 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:31.270589    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6335aded-6553-4a60-af1e-1c835ccc3ba5","Type":"ContainerStarted","Data":"0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926"}
Oct 30 10:54:31.314732 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:31.314686    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.345182566 podStartE2EDuration="12.314674431s" totalImagesPullingTime="10.969491865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:54:31.313813955 +0000 UTC m=+568.694263393" watchObservedRunningTime="2026-10-30 10:54:31.314674431 +0000 UTC m=+568.695123869"
Oct 30 10:54:32.277927 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:32.277885    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerStarted","Data":"2eba01aff7128ff490deb051e16afd154b7f0642413c0335af5854580fe493eb"}
Oct 30 10:54:33.284830 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:33.284799    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerStarted","Data":"bed8d7be25f214e9d434c8fb4d1b973b9cb37b8c4c00b87920eeb7debbd91bc7"}
Oct 30 10:54:33.285227 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:33.284912    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:33.285227 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:33.285053    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:33.285227 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:33.285086    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:33.287463 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:33.287438    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:33.293488 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:33.293472    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:33.363104 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:33.363068    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podStartSLOduration=8.0468139 podStartE2EDuration="13.363057509s" totalImagesPullingTime="5.316243609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:54:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:54:33.34797304 +0000 UTC m=+570.728422482" watchObservedRunningTime="2026-10-30 10:54:33.363057509 +0000 UTC m=+570.743506947"
Oct 30 10:54:37.633414 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:37.633380    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:37.633848 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:37.633583    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="6335aded-6553-4a60-af1e-1c835ccc3ba5" containerName="otc-container" containerID="cri-o://0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926" gracePeriod=30
Oct 30 10:54:37.892443 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:37.892387    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:38.008660 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.008629    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6335aded-6553-4a60-af1e-1c835ccc3ba5-otc-internal\" (UniqueName: \"kubernetes.io/configmap/6335aded-6553-4a60-af1e-1c835ccc3ba5-otc-internal\") pod \"6335aded-6553-4a60-af1e-1c835ccc3ba5\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") "
Oct 30 10:54:38.008800 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.008680    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6335aded-6553-4a60-af1e-1c835ccc3ba5-kube-api-access-mzb9k\" (UniqueName: \"kubernetes.io/projected/6335aded-6553-4a60-af1e-1c835ccc3ba5-kube-api-access-mzb9k\") pod \"6335aded-6553-4a60-af1e-1c835ccc3ba5\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") "
Oct 30 10:54:38.008800 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.008710    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6335aded-6553-4a60-af1e-1c835ccc3ba5-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6335aded-6553-4a60-af1e-1c835ccc3ba5-monitoring-tls-certs\") pod \"6335aded-6553-4a60-af1e-1c835ccc3ba5\" (UID: \"6335aded-6553-4a60-af1e-1c835ccc3ba5\") "
Oct 30 10:54:38.009072 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.009042    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6335aded-6553-4a60-af1e-1c835ccc3ba5-otc-internal" pod "6335aded-6553-4a60-af1e-1c835ccc3ba5" (UID: "6335aded-6553-4a60-af1e-1c835ccc3ba5"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:54:38.010793 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.010769    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6335aded-6553-4a60-af1e-1c835ccc3ba5-monitoring-tls-certs" pod "6335aded-6553-4a60-af1e-1c835ccc3ba5" (UID: "6335aded-6553-4a60-af1e-1c835ccc3ba5"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:54:38.010870 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.010834    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6335aded-6553-4a60-af1e-1c835ccc3ba5-kube-api-access-mzb9k" pod "6335aded-6553-4a60-af1e-1c835ccc3ba5" (UID: "6335aded-6553-4a60-af1e-1c835ccc3ba5"). InnerVolumeSpecName "kube-api-access-mzb9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:54:38.067681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.067650    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:54:38.067994 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.067967    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo" containerID="cri-o://7c497d2458a7ab7d218512fc08ca1206f71358b396d4b578c7f9b18a1169c767" gracePeriod=30
Oct 30 10:54:38.068170 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.068048    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway-opa" containerID="cri-o://bed8d7be25f214e9d434c8fb4d1b973b9cb37b8c4c00b87920eeb7debbd91bc7" gracePeriod=30
Oct 30 10:54:38.068280 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.068161    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway" containerID="cri-o://2eba01aff7128ff490deb051e16afd154b7f0642413c0335af5854580fe493eb" gracePeriod=30
Oct 30 10:54:38.109393 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.109369    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6335aded-6553-4a60-af1e-1c835ccc3ba5-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:38.109485 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.109400    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzb9k\" (UniqueName: \"kubernetes.io/projected/6335aded-6553-4a60-af1e-1c835ccc3ba5-kube-api-access-mzb9k\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:38.109485 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.109412    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6335aded-6553-4a60-af1e-1c835ccc3ba5-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:38.285929 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.285888    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway-opa" probeResult="failure" output="Get \"http://10.132.0.39:8083/ready\": dial tcp 10.132.0.39:8083: connect: connection refused"
Oct 30 10:54:38.289631 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.289606    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 10:54:38.317856 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.317833    4849 generic.go:356] "Generic (PLEG): container finished" podID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerID="bed8d7be25f214e9d434c8fb4d1b973b9cb37b8c4c00b87920eeb7debbd91bc7" exitCode=0
Oct 30 10:54:38.317856 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.317856    4849 generic.go:356] "Generic (PLEG): container finished" podID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerID="7c497d2458a7ab7d218512fc08ca1206f71358b396d4b578c7f9b18a1169c767" exitCode=0
Oct 30 10:54:38.318018 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.317904    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerDied","Data":"bed8d7be25f214e9d434c8fb4d1b973b9cb37b8c4c00b87920eeb7debbd91bc7"}
Oct 30 10:54:38.318018 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.317954    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerDied","Data":"7c497d2458a7ab7d218512fc08ca1206f71358b396d4b578c7f9b18a1169c767"}
Oct 30 10:54:38.319263 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.319242    4849 generic.go:356] "Generic (PLEG): container finished" podID="6335aded-6553-4a60-af1e-1c835ccc3ba5" containerID="0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926" exitCode=0
Oct 30 10:54:38.319352 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.319290    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:38.319352 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.319295    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6335aded-6553-4a60-af1e-1c835ccc3ba5","Type":"ContainerDied","Data":"0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926"}
Oct 30 10:54:38.319352 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.319342    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6335aded-6553-4a60-af1e-1c835ccc3ba5","Type":"ContainerDied","Data":"354624cca07e1535707cc0b5c79ce74f2fe98ec99b7c0f868bfb2fa23e93ee81"}
Oct 30 10:54:38.319515 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.319374    4849 scope.go:122] "RemoveContainer" containerID="0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926"
Oct 30 10:54:38.327528 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.327506    4849 scope.go:122] "RemoveContainer" containerID="0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926"
Oct 30 10:54:38.327774 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:54:38.327755    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926\": container with ID starting with 0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926 not found: ID does not exist" containerID="0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926"
Oct 30 10:54:38.327830 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.327783    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926"} err="failed to get container status \"0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926\": rpc error: code = NotFound desc = could not find container \"0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926\": container with ID starting with 0ae6afab2091546fb0c10abf8855121ca1d1bb70c2570fc1b78041bb7fa6d926 not found: ID does not exist"
Oct 30 10:54:38.373671 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.373646    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:38.378031 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.378000    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:38.753666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:38.753638    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6335aded-6553-4a60-af1e-1c835ccc3ba5" path="/var/lib/kubelet/pods/6335aded-6553-4a60-af1e-1c835ccc3ba5/volumes"
Oct 30 10:54:39.327030 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.327000    4849 generic.go:356] "Generic (PLEG): container finished" podID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerID="2eba01aff7128ff490deb051e16afd154b7f0642413c0335af5854580fe493eb" exitCode=0
Oct 30 10:54:39.327153 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.327050    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerDied","Data":"2eba01aff7128ff490deb051e16afd154b7f0642413c0335af5854580fe493eb"}
Oct 30 10:54:39.327153 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.327106    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"25a38f5d-75a5-4b8b-9f22-f475046a138a","Type":"ContainerDied","Data":"ee65da1a02becc6ad9f6b68bfed7936661405db59ea1e767611d2d3ac4ca392e"}
Oct 30 10:54:39.327153 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.327126    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee65da1a02becc6ad9f6b68bfed7936661405db59ea1e767611d2d3ac4ca392e"
Oct 30 10:54:39.328805 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.328789    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:39.421612 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421554    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25a38f5d-75a5-4b8b-9f22-f475046a138a-kube-api-access-vddgs\" (UniqueName: \"kubernetes.io/projected/25a38f5d-75a5-4b8b-9f22-f475046a138a-kube-api-access-vddgs\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421618    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-tenants\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-tenants\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421655    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-conf\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421812 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421722    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421812 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421744    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421812 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421786    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-ca-bundle\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421947 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421831    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421947 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421865    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-mtls\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.421947 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.421891    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-rbac\") pod \"25a38f5d-75a5-4b8b-9f22-f475046a138a\" (UID: \"25a38f5d-75a5-4b8b-9f22-f475046a138a\") "
Oct 30 10:54:39.422125 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.422090    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-conf" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:54:39.422247 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.422191    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.422742 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.422563    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-serving-cabundle" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-serving-cabundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:54:39.422742 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.422678    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-rbac" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "gateway-rbac". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:54:39.424088 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.423299    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-ca-bundle" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:54:39.425228 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.425202    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a38f5d-75a5-4b8b-9f22-f475046a138a-kube-api-access-vddgs" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "kube-api-access-vddgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:54:39.425315 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.425266    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-tenants" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "gateway-tenants". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:54:39.425421 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.425402    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-mtls" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:54:39.427002 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.426977    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-gateway-serving-cert" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-gateway-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:54:39.427444 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.427422    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e" pod "25a38f5d-75a5-4b8b-9f22-f475046a138a" (UID: "25a38f5d-75a5-4b8b-9f22-f475046a138a"). InnerVolumeSpecName "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 10:54:39.432016 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:54:39.431989    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:54:39.522836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522770    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vddgs\" (UniqueName: \"kubernetes.io/projected/25a38f5d-75a5-4b8b-9f22-f475046a138a-kube-api-access-vddgs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.522836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522790    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-tenants\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.522836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522816    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 10:54:39.522836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522826    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-serving-cabundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.522836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522836    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.543777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522846    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-gateway-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.543777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522855    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/25a38f5d-75a5-4b8b-9f22-f475046a138a-tempo-data-science-tempomonolithic-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.543777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.522863    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/25a38f5d-75a5-4b8b-9f22-f475046a138a-gateway-rbac\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:54:39.543777 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.533612    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e") on node "ip-10-0-140-230.ec2.internal"
Oct 30 10:54:39.623501 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:39.623481    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\""
Oct 30 10:54:40.333870 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:40.333838    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:54:40.403052 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:40.403010    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:54:40.416619 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:40.416589    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:54:40.753469 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:40.753406    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" path="/var/lib/kubelet/pods/25a38f5d-75a5-4b8b-9f22-f475046a138a/volumes"
Oct 30 10:54:44.799391 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799313    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:44.799862 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799760    4849 cpu_manager.go:414] "containerMap: removing container" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo"
Oct 30 10:54:44.799862 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799773    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo"
Oct 30 10:54:44.799862 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799819    4849 cpu_manager.go:414] "containerMap: removing container" podUID="6335aded-6553-4a60-af1e-1c835ccc3ba5" containerName="otc-container"
Oct 30 10:54:44.799862 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799828    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6335aded-6553-4a60-af1e-1c835ccc3ba5" containerName="otc-container"
Oct 30 10:54:44.799862 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799840    4849 cpu_manager.go:414] "containerMap: removing container" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway"
Oct 30 10:54:44.799862 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799847    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway"
Oct 30 10:54:44.800206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799870    4849 cpu_manager.go:414] "containerMap: removing container" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway-opa"
Oct 30 10:54:44.800206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799877    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway-opa"
Oct 30 10:54:44.800206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799958    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="6335aded-6553-4a60-af1e-1c835ccc3ba5" containerName="otc-container"
Oct 30 10:54:44.800206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799969    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway"
Oct 30 10:54:44.800206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799982    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo-gateway-opa"
Oct 30 10:54:44.800206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.799992    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a38f5d-75a5-4b8b-9f22-f475046a138a" containerName="tempo"
Oct 30 10:54:44.807081 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.807055    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:44.825978 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.825950    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:44.826700 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.826679    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:54:44.837067 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.837017    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:54:44.837541 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.837525    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:54:44.840350 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.840329    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 10:54:44.842632 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.842609    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 10:54:44.842632 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.842623    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:54:44.969439 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.969400    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:44.969645 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.969479    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:44.969645 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.969514    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:44.969645 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.969576    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:44.969645 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:44.969607    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp8r9\" (UniqueName: \"kubernetes.io/projected/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-kube-api-access-pp8r9\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.070549 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.070520    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.070679 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.070561    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.070679 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.070579    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.070679 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.070629    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.070679 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.070659    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp8r9\" (UniqueName: \"kubernetes.io/projected/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-kube-api-access-pp8r9\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.070857 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:54:45.070689    4849 secret.go:189] Couldn't get secret redhat-ods-monitoring/data-science-collector-ta-client-cert: secret "data-science-collector-ta-client-cert" not found
Oct 30 10:54:45.070857 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:54:45.070761    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert podName:d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2 nodeName:}" failed. No retries permitted until 2026-10-30 10:54:45.570739904 +0000 UTC m=+582.951189332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "data-science-collector-ta-client-cert" (UniqueName: "kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert") pod "data-science-collector-collector-0" (UID: "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2") : secret "data-science-collector-ta-client-cert" not found
Oct 30 10:54:45.071501 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.071476    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.073040 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.073008    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.073146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.073036    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.099209 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.099187    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp8r9\" (UniqueName: \"kubernetes.io/projected/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-kube-api-access-pp8r9\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.575744 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:45.575717    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:45.575910 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:54:45.575824    4849 secret.go:189] Couldn't get secret redhat-ods-monitoring/data-science-collector-ta-client-cert: secret "data-science-collector-ta-client-cert" not found
Oct 30 10:54:45.575910 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:54:45.575874    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert podName:d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2 nodeName:}" failed. No retries permitted until 2026-10-30 10:54:46.57586017 +0000 UTC m=+583.956309588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "data-science-collector-ta-client-cert" (UniqueName: "kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert") pod "data-science-collector-collector-0" (UID: "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2") : secret "data-science-collector-ta-client-cert" not found
Oct 30 10:54:46.585214 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:46.585181    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:46.587635 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:46.587608    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:46.620573 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:46.620550    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:54:46.929498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:46.929471    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:54:46.931383 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:54:46.931356    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ef6236_b1d3_4b2a_b85c_694d2af1f9e2.slice/crio-4c0671d5f388b8519f1fb06784882b10652fb8d1b9c15774132f971fd8a400c4 WatchSource:0}: Error finding container 4c0671d5f388b8519f1fb06784882b10652fb8d1b9c15774132f971fd8a400c4: Status 404 returned error can't find the container with id 4c0671d5f388b8519f1fb06784882b10652fb8d1b9c15774132f971fd8a400c4
Oct 30 10:54:46.933432 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:46.933397    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:54:46.933514 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:46.933459    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:54:47.370682 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:47.370651    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2","Type":"ContainerStarted","Data":"5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b"}
Oct 30 10:54:47.370848 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:47.370701    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2","Type":"ContainerStarted","Data":"4c0671d5f388b8519f1fb06784882b10652fb8d1b9c15774132f971fd8a400c4"}
Oct 30 10:54:47.416664 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:47.416612    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=3.416599224 podStartE2EDuration="3.416599224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:54:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:54:47.416046232 +0000 UTC m=+584.796495672" watchObservedRunningTime="2026-10-30 10:54:47.416599224 +0000 UTC m=+584.797048662"
Oct 30 10:54:48.165858 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.165824    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 10:54:48.199402 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.199378    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 10:54:48.199535 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.199481    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.216738 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.216711    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 10:54:48.216738 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.216711    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 10:54:48.216940 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.216722    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 10:54:48.216940 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.216754    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 10:54:48.216940 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.216794    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-vfhvl\""
Oct 30 10:54:48.301825 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.301797    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.301965 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.301836    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.301965 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.301877    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.302099 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.302016    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.302099 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.302082    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.302182 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.302111    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.302237 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.302210    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8trw\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-kube-api-access-c8trw\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.403271 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.403239    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.403474 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.403281    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8trw\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-kube-api-access-c8trw\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.403474 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.403325    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.403474 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.403345    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.403474 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.403378    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.403474 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.403408    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.403474 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.403441    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.404135 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.404075    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.406043 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.406001    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.406264 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.406240    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.406347 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.406332    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.406676 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.406654    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.407127 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.407108    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.434455 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.434404    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8trw\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-kube-api-access-c8trw\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.563429 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.563406    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:48.919861 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:48.919833    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 10:54:48.936263 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:54:48.936218    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf960743_2f84_4069_8a1d_0c8e65af4d9b.slice/crio-1f9aa7485d5107d334854ce485f264660005501bc526c45fb7d4935da4a73c06 WatchSource:0}: Error finding container 1f9aa7485d5107d334854ce485f264660005501bc526c45fb7d4935da4a73c06: Status 404 returned error can't find the container with id 1f9aa7485d5107d334854ce485f264660005501bc526c45fb7d4935da4a73c06
Oct 30 10:54:49.382975 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:49.382941    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerStarted","Data":"1f9aa7485d5107d334854ce485f264660005501bc526c45fb7d4935da4a73c06"}
Oct 30 10:54:51.394690 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.394651    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerStarted","Data":"9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6"}
Oct 30 10:54:51.479217 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.479187    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 10:54:51.482079 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.482060    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.498222 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498198    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 10:54:51.498719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498691    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 10:54:51.498719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498710    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 10:54:51.498897 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498710    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 10:54:51.498897 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498714    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 10:54:51.498897 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498716    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 10:54:51.498897 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498749    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 10:54:51.498897 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498815    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 10:54:51.498897 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498877    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-jh8zq\""
Oct 30 10:54:51.499123 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.498952    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 10:54:51.513792 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.513769    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 10:54:51.629451 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629425    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629458    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629482    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629635 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629553    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629635 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629606    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629698 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629639    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629706    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbrmv\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-kube-api-access-gbrmv\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629771 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629737    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629818 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629804    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629852 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629828    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629849    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.629882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.629865    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731235 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731172    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731235 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731222    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731403 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731254    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbrmv\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-kube-api-access-gbrmv\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731403 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731359    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731522 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731406    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731522 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731434    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731522 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731464    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731522 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731494    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731539    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731570    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731618    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.731719 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.731645    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.732209 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.732184    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.732312 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.732213    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.732754 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.732730    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.734307 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.734278    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.734406 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.734383    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.734624 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.734583    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.734725 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.734657    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.734998 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.734977    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.735064 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.735050    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.735109 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.735091    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.735268 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.735251    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:51.755551 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:51.755530    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbrmv\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-kube-api-access-gbrmv\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:54.410627 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:54.410592    4849 generic.go:356] "Generic (PLEG): container finished" podID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerID="9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6" exitCode=0
Oct 30 10:54:54.411011 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:54.410662    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerDied","Data":"9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6"}
Oct 30 10:54:55.459220 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.459180    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:55.490426 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.490395    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:55.560463 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.560431    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:55.560615 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.560585    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:55.576242 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.576216    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:55.756312 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.756278    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5f4e377c3e259c071aab429e31ca61846b879cc3476e2f02b70e888ccdcdb590/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:55.783151 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.783122    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:55.997305 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:55.997129    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:54:56.353054 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:56.353011    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 10:54:56.356214 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:54:56.356189    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef541574_0c7d_4a58_b3fb_4cf1f820217c.slice/crio-849cf0ddbced9405aba93026c85f4610f2c39a113a0ea9ae0f64336d678df884 WatchSource:0}: Error finding container 849cf0ddbced9405aba93026c85f4610f2c39a113a0ea9ae0f64336d678df884: Status 404 returned error can't find the container with id 849cf0ddbced9405aba93026c85f4610f2c39a113a0ea9ae0f64336d678df884
Oct 30 10:54:56.423205 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:56.423182    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerStarted","Data":"849cf0ddbced9405aba93026c85f4610f2c39a113a0ea9ae0f64336d678df884"}
Oct 30 10:54:57.433882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:57.433850    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerStarted","Data":"42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda"}
Oct 30 10:54:57.434323 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:57.433911    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerStarted","Data":"ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee"}
Oct 30 10:54:57.434323 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:57.434153    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:54:57.435651 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:57.435629    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerStarted","Data":"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"}
Oct 30 10:54:57.510971 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:54:57.510934    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podStartSLOduration=3.026049806 podStartE2EDuration="9.510922982s" totalImagesPullingTime="3.294327439s" totalInitContainerRuntime="3.190545737s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:54:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:54:57.510199577 +0000 UTC m=+594.890649015" watchObservedRunningTime="2026-10-30 10:54:57.510922982 +0000 UTC m=+594.891372464"
Oct 30 10:55:00.455371 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:00.455295    4849 generic.go:356] "Generic (PLEG): container finished" podID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerID="2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa" exitCode=0
Oct 30 10:55:00.455371 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:00.455358    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerDied","Data":"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"}
Oct 30 10:55:02.804244 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:02.804215    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 10:55:02.804800 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:02.804780    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 10:55:03.443252 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:03.443222    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 10:55:04.479924 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:04.479852    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerStarted","Data":"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"}
Oct 30 10:55:04.480309 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:04.479929    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerStarted","Data":"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"}
Oct 30 10:55:06.492999 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:06.492964    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerStarted","Data":"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"}
Oct 30 10:55:06.590319 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:06.590273    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podStartSLOduration=10.560734367 podStartE2EDuration="18.59025913s" totalImagesPullingTime="4.840934386s" totalInitContainerRuntime="3.188590377s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:54:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:55:06.574513349 +0000 UTC m=+603.954962788" watchObservedRunningTime="2026-10-30 10:55:06.59025913 +0000 UTC m=+603.970708568"
Oct 30 10:55:09.433207 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:55:09.433165    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:55:10.997921 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:10.997885    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:55:10.998338 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:10.997958    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:55:11.004042 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:11.004000    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:55:11.526856 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:11.526827    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 10:55:23.127061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:23.127012    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:55:23.127808 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:23.127756    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" containerName="otc-container" containerID="cri-o://5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b" gracePeriod=30
Oct 30 10:55:25.472882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.472857    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.591635 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.591607    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-kube-api-access-pp8r9\" (UniqueName: \"kubernetes.io/projected/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-kube-api-access-pp8r9\") pod \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") "
Oct 30 10:55:25.591771 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.591677    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-tls-certs\") pod \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") "
Oct 30 10:55:25.591771 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.591709    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-monitoring-tls-certs\") pod \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") "
Oct 30 10:55:25.591863 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.591775    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\") pod \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") "
Oct 30 10:55:25.591863 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.591815    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-otc-internal\") pod \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\" (UID: \"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2\") "
Oct 30 10:55:25.592381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.592341    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-otc-internal" pod "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" (UID: "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:55:25.593933 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.593893    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-tls-certs" pod "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" (UID: "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:55:25.594062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.594014    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-monitoring-tls-certs" pod "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" (UID: "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:55:25.594172 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.594149    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert" pod "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" (UID: "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:55:25.595174 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.595149    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-kube-api-access-pp8r9" pod "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" (UID: "d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2"). InnerVolumeSpecName "kube-api-access-pp8r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:55:25.595256 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.595190    4849 generic.go:356] "Generic (PLEG): container finished" podID="d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" containerID="5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b" exitCode=0
Oct 30 10:55:25.595295 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.595263    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.595295 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.595284    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2","Type":"ContainerDied","Data":"5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b"}
Oct 30 10:55:25.595393 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.595328    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2","Type":"ContainerDied","Data":"4c0671d5f388b8519f1fb06784882b10652fb8d1b9c15774132f971fd8a400c4"}
Oct 30 10:55:25.595393 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.595351    4849 scope.go:122] "RemoveContainer" containerID="5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b"
Oct 30 10:55:25.613232 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.613215    4849 scope.go:122] "RemoveContainer" containerID="5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b"
Oct 30 10:55:25.613512 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:55:25.613494    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b\": container with ID starting with 5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b not found: ID does not exist" containerID="5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b"
Oct 30 10:55:25.613562 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.613519    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b"} err="failed to get container status \"5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b\": rpc error: code = NotFound desc = could not find container \"5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b\": container with ID starting with 5831a0f2fc3afbfcd316ff5b9d78f415e8f685b78b49a01bc6ce886a226f195b not found: ID does not exist"
Oct 30 10:55:25.649745 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.649725    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:55:25.653134 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.653116    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:55:25.689289 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.689264    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:55:25.689618 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.689606    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" containerName="otc-container"
Oct 30 10:55:25.689661 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.689619    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" containerName="otc-container"
Oct 30 10:55:25.689702 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.689694    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" containerName="otc-container"
Oct 30 10:55:25.692393 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.692366    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:55:25.692393 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.692396    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:55:25.692490 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.692407    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp8r9\" (UniqueName: \"kubernetes.io/projected/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-kube-api-access-pp8r9\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:55:25.692490 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.692416    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:55:25.692490 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.692425    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:55:25.694736 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.694722    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.711349 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.711330    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:55:25.711666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.711550    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:55:25.711666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.711598    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:55:25.711801 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.711621    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:55:25.711801 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.711646    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:55:25.713320 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.713299    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:55:25.793163 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.793139    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tcrr\" (UniqueName: \"kubernetes.io/projected/70372a65-340d-4251-9e1a-93cc19308e41-kube-api-access-6tcrr\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.793277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.793198    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.793331 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.793313    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/70372a65-340d-4251-9e1a-93cc19308e41-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.793385 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.793373    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.793431 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.793391    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.894617 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.894565    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/70372a65-340d-4251-9e1a-93cc19308e41-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.894617 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.894601    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.894778 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.894619    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.894778 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.894737    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tcrr\" (UniqueName: \"kubernetes.io/projected/70372a65-340d-4251-9e1a-93cc19308e41-kube-api-access-6tcrr\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.894877 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.894786    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.895246 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.895226    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/70372a65-340d-4251-9e1a-93cc19308e41-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.897178 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.897160    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.897304 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.897283    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.897341 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.897291    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:25.919043 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:25.919011    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tcrr\" (UniqueName: \"kubernetes.io/projected/70372a65-340d-4251-9e1a-93cc19308e41-kube-api-access-6tcrr\") pod \"data-science-collector-collector-0\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:26.007486 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.007459    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:55:26.307791 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.307769    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:55:26.309843 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:55:26.309811    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70372a65_340d_4251_9e1a_93cc19308e41.slice/crio-3613042a96e7aa46ecd4556ea74441841e3d7230d6a2a4e3c8d163c8ea26dd0f WatchSource:0}: Error finding container 3613042a96e7aa46ecd4556ea74441841e3d7230d6a2a4e3c8d163c8ea26dd0f: Status 404 returned error can't find the container with id 3613042a96e7aa46ecd4556ea74441841e3d7230d6a2a4e3c8d163c8ea26dd0f
Oct 30 10:55:26.312003 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.311974    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:55:26.312110 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.312051    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:55:26.602755 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.602683    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"70372a65-340d-4251-9e1a-93cc19308e41","Type":"ContainerStarted","Data":"dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531"}
Oct 30 10:55:26.602755 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.602733    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"70372a65-340d-4251-9e1a-93cc19308e41","Type":"ContainerStarted","Data":"3613042a96e7aa46ecd4556ea74441841e3d7230d6a2a4e3c8d163c8ea26dd0f"}
Oct 30 10:55:26.647226 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.647176    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.647162341 podStartE2EDuration="1.647162341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:55:26.645784917 +0000 UTC m=+624.026234356" watchObservedRunningTime="2026-10-30 10:55:26.647162341 +0000 UTC m=+624.027611847"
Oct 30 10:55:26.752762 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:55:26.752730    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2" path="/var/lib/kubelet/pods/d0ef6236-b1d3-4b2a-b85c-694d2af1f9e2/volumes"
Oct 30 10:55:39.431259 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:55:39.431222    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:56:09.432122 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:56:09.432079    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:56:39.431923 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:56:39.431841    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:56:49.309206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.309173    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:56:49.313067 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.313047    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.329671 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.329649    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:56:49.343454 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.343432    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-dockercfg-t6jph\""
Oct 30 10:56:49.343897 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.343873    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway-serving-cert\""
Oct 30 10:56:49.344050 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.344014    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-ca-bundle\""
Oct 30 10:56:49.344129 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.344096    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:56:49.344190 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.344163    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-mtls\""
Oct 30 10:56:49.344574 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.344545    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-config\""
Oct 30 10:56:49.344703 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.344615    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-serving-cabundle\""
Oct 30 10:56:49.346819 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.346801    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:56:49.444374 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444353    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.444480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444388    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.444480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444413    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.444576 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444549    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.444631 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444588    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.444631 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444609    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.444631 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444627    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zv7v\" (UniqueName: \"kubernetes.io/projected/625b195e-5962-4216-aaf5-a5f257910389-kube-api-access-2zv7v\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.444795 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.444735    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545205 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545182    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545310 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545217    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545310 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545237    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545310 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545260    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zv7v\" (UniqueName: \"kubernetes.io/projected/625b195e-5962-4216-aaf5-a5f257910389-kube-api-access-2zv7v\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545310 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545290    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545505 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545353    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545505 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545390    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.545505 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545423    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.546014 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.545988    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.546211 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.546136    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.546324 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.546249    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.546324 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.546273    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.547947 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.547924    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.548297 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.548278    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.548348 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.548284    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:49.569883 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:49.569835    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zv7v\" (UniqueName: \"kubernetes.io/projected/625b195e-5962-4216-aaf5-a5f257910389-kube-api-access-2zv7v\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:52.533277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:52.533235    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:52.533812 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:52.533554    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="70372a65-340d-4251-9e1a-93cc19308e41" containerName="otc-container" containerID="cri-o://dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531" gracePeriod=30
Oct 30 10:56:54.881281 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.881259    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:54.989511 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.989438    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-monitoring-tls-certs\") pod \"70372a65-340d-4251-9e1a-93cc19308e41\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") "
Oct 30 10:56:54.989511 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.989480    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-data-science-collector-ta-client-cert\") pod \"70372a65-340d-4251-9e1a-93cc19308e41\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") "
Oct 30 10:56:54.989704 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.989522    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70372a65-340d-4251-9e1a-93cc19308e41-otc-internal\" (UniqueName: \"kubernetes.io/configmap/70372a65-340d-4251-9e1a-93cc19308e41-otc-internal\") pod \"70372a65-340d-4251-9e1a-93cc19308e41\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") "
Oct 30 10:56:54.989704 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.989553    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-tls-certs\") pod \"70372a65-340d-4251-9e1a-93cc19308e41\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") "
Oct 30 10:56:54.989704 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.989678    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70372a65-340d-4251-9e1a-93cc19308e41-kube-api-access-6tcrr\" (UniqueName: \"kubernetes.io/projected/70372a65-340d-4251-9e1a-93cc19308e41-kube-api-access-6tcrr\") pod \"70372a65-340d-4251-9e1a-93cc19308e41\" (UID: \"70372a65-340d-4251-9e1a-93cc19308e41\") "
Oct 30 10:56:54.989899 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.989872    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70372a65-340d-4251-9e1a-93cc19308e41-otc-internal" pod "70372a65-340d-4251-9e1a-93cc19308e41" (UID: "70372a65-340d-4251-9e1a-93cc19308e41"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:56:54.991655 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.991621    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70372a65-340d-4251-9e1a-93cc19308e41-kube-api-access-6tcrr" pod "70372a65-340d-4251-9e1a-93cc19308e41" (UID: "70372a65-340d-4251-9e1a-93cc19308e41"). InnerVolumeSpecName "kube-api-access-6tcrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:56:54.991765 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.991660    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-data-science-collector-ta-client-cert" pod "70372a65-340d-4251-9e1a-93cc19308e41" (UID: "70372a65-340d-4251-9e1a-93cc19308e41"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:54.991765 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.991680    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-tls-certs" pod "70372a65-340d-4251-9e1a-93cc19308e41" (UID: "70372a65-340d-4251-9e1a-93cc19308e41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:54.991765 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:54.991737    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-monitoring-tls-certs" pod "70372a65-340d-4251-9e1a-93cc19308e41" (UID: "70372a65-340d-4251-9e1a-93cc19308e41"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:55.074638 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.074614    4849 generic.go:356] "Generic (PLEG): container finished" podID="70372a65-340d-4251-9e1a-93cc19308e41" containerID="dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531" exitCode=0
Oct 30 10:56:55.074763 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.074661    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.074763 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.074704    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"70372a65-340d-4251-9e1a-93cc19308e41","Type":"ContainerDied","Data":"dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531"}
Oct 30 10:56:55.074880 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.074765    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"70372a65-340d-4251-9e1a-93cc19308e41","Type":"ContainerDied","Data":"3613042a96e7aa46ecd4556ea74441841e3d7230d6a2a4e3c8d163c8ea26dd0f"}
Oct 30 10:56:55.074880 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.074796    4849 scope.go:122] "RemoveContainer" containerID="dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531"
Oct 30 10:56:55.083742 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.083722    4849 scope.go:122] "RemoveContainer" containerID="dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531"
Oct 30 10:56:55.084005 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:56:55.083985    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531\": container with ID starting with dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531 not found: ID does not exist" containerID="dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531"
Oct 30 10:56:55.084125 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.084012    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531"} err="failed to get container status \"dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531\": rpc error: code = NotFound desc = could not find container \"dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531\": container with ID starting with dc465e19db7eed305476fe475e2bbc175f97bc881449e6c5cc14c6c2caaf0531 not found: ID does not exist"
Oct 30 10:56:55.090468 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.090446    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tcrr\" (UniqueName: \"kubernetes.io/projected/70372a65-340d-4251-9e1a-93cc19308e41-kube-api-access-6tcrr\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:55.090559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.090472    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:55.090559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.090488    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:55.090559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.090503    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/70372a65-340d-4251-9e1a-93cc19308e41-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:55.090559 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.090516    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70372a65-340d-4251-9e1a-93cc19308e41-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:55.133160 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.133124    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:55.148262 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.148239    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:55.185469 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.185450    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:55.185867 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.185850    4849 cpu_manager.go:414] "containerMap: removing container" podUID="70372a65-340d-4251-9e1a-93cc19308e41" containerName="otc-container"
Oct 30 10:56:55.185949 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.185869    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70372a65-340d-4251-9e1a-93cc19308e41" containerName="otc-container"
Oct 30 10:56:55.186054 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.186041    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="70372a65-340d-4251-9e1a-93cc19308e41" containerName="otc-container"
Oct 30 10:56:55.190398 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.190379    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.210953 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.210922    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:55.211069 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.211014    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:56:55.211069 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.211049    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:56:55.211175 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.211156    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:56:55.221444 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.221424    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-7c92b5b1\""
Oct 30 10:56:55.221524 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.221454    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:56:55.292006 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.291955    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.292096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.292040    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.292096 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.292068    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.292170 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.292098    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5239e678-35b6-4e45-bee5-aeebbf26d5a4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.292170 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.292139    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x8j7\" (UniqueName: \"kubernetes.io/projected/5239e678-35b6-4e45-bee5-aeebbf26d5a4-kube-api-access-5x8j7\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.392779 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.392751    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.392888 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.392784    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.392888 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.392816    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5239e678-35b6-4e45-bee5-aeebbf26d5a4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.392888 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.392838    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x8j7\" (UniqueName: \"kubernetes.io/projected/5239e678-35b6-4e45-bee5-aeebbf26d5a4-kube-api-access-5x8j7\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.393073 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.392888    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.393603 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.393583    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5239e678-35b6-4e45-bee5-aeebbf26d5a4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.395286 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.395266    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.395367 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.395355    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.395411 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.395367    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.417215 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.417194    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x8j7\" (UniqueName: \"kubernetes.io/projected/5239e678-35b6-4e45-bee5-aeebbf26d5a4-kube-api-access-5x8j7\") pod \"data-science-collector-collector-0\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.502712 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.502687    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:55.801203 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.801159    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:55.803402 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:56:55.803378    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5239e678_35b6_4e45_bee5_aeebbf26d5a4.slice/crio-a2c8657ba93f248bec4ce0ce8481f711a6416cb98eb3448bd216379337a51d1d WatchSource:0}: Error finding container a2c8657ba93f248bec4ce0ce8481f711a6416cb98eb3448bd216379337a51d1d: Status 404 returned error can't find the container with id a2c8657ba93f248bec4ce0ce8481f711a6416cb98eb3448bd216379337a51d1d
Oct 30 10:56:55.805617 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.805587    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:56:55.805687 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:55.805645    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:56:56.083464 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:56.083430    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5239e678-35b6-4e45-bee5-aeebbf26d5a4","Type":"ContainerStarted","Data":"1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212"}
Oct 30 10:56:56.083837 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:56.083559    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5239e678-35b6-4e45-bee5-aeebbf26d5a4","Type":"ContainerStarted","Data":"a2c8657ba93f248bec4ce0ce8481f711a6416cb98eb3448bd216379337a51d1d"}
Oct 30 10:56:56.129784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:56.129718    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.129704807 podStartE2EDuration="1.129704807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:56:56.127359112 +0000 UTC m=+713.507808552" watchObservedRunningTime="2026-10-30 10:56:56.129704807 +0000 UTC m=+713.510154246"
Oct 30 10:56:56.753150 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:56.753118    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70372a65-340d-4251-9e1a-93cc19308e41" path="/var/lib/kubelet/pods/70372a65-340d-4251-9e1a-93cc19308e41/volumes"
Oct 30 10:56:57.617643 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:57.617609    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:57.649427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:57.649396    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") device path: \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:57.718270 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:57.718237    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:57.718395 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:57.718351    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") DevicePath \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:57.733970 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:57.733946    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:57.837695 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:57.837674    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/fd3ed81014d76a975b5ba6f044c00684e55bfb3daddf6ff0e225ceff74c57792/globalmount\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:57.865447 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:57.865423    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:58.030498 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.030443    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:58.170213 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.170183    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:56:58.348502 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.348478    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:56:58.350298 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:56:58.350264    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625b195e_5962_4216_aaf5_a5f257910389.slice/crio-8d8470922c19fb368288d0cb7135ba82673099dd1516a8559422fd93e266879b WatchSource:0}: Error finding container 8d8470922c19fb368288d0cb7135ba82673099dd1516a8559422fd93e266879b: Status 404 returned error can't find the container with id 8d8470922c19fb368288d0cb7135ba82673099dd1516a8559422fd93e266879b
Oct 30 10:56:58.583654 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.583620    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:58.584048 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.583824    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="5239e678-35b6-4e45-bee5-aeebbf26d5a4" containerName="otc-container" containerID="cri-o://1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212" gracePeriod=30
Oct 30 10:56:58.840954 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.840930    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:58.929336 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.929263    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-tls-certs\") pod \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") "
Oct 30 10:56:58.929336 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.929297    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-monitoring-tls-certs\") pod \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") "
Oct 30 10:56:58.929491 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.929340    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5239e678-35b6-4e45-bee5-aeebbf26d5a4-otc-internal\" (UniqueName: \"kubernetes.io/configmap/5239e678-35b6-4e45-bee5-aeebbf26d5a4-otc-internal\") pod \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") "
Oct 30 10:56:58.929491 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.929443    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-data-science-collector-ta-client-cert\") pod \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") "
Oct 30 10:56:58.929491 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.929472    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5239e678-35b6-4e45-bee5-aeebbf26d5a4-kube-api-access-5x8j7\" (UniqueName: \"kubernetes.io/projected/5239e678-35b6-4e45-bee5-aeebbf26d5a4-kube-api-access-5x8j7\") pod \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\" (UID: \"5239e678-35b6-4e45-bee5-aeebbf26d5a4\") "
Oct 30 10:56:58.929763 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.929743    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5239e678-35b6-4e45-bee5-aeebbf26d5a4-otc-internal" pod "5239e678-35b6-4e45-bee5-aeebbf26d5a4" (UID: "5239e678-35b6-4e45-bee5-aeebbf26d5a4"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:56:58.931471 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.931443    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5239e678-35b6-4e45-bee5-aeebbf26d5a4-kube-api-access-5x8j7" pod "5239e678-35b6-4e45-bee5-aeebbf26d5a4" (UID: "5239e678-35b6-4e45-bee5-aeebbf26d5a4"). InnerVolumeSpecName "kube-api-access-5x8j7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:56:58.932006 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.931980    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-tls-certs" pod "5239e678-35b6-4e45-bee5-aeebbf26d5a4" (UID: "5239e678-35b6-4e45-bee5-aeebbf26d5a4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:58.932119 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.932094    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-monitoring-tls-certs" pod "5239e678-35b6-4e45-bee5-aeebbf26d5a4" (UID: "5239e678-35b6-4e45-bee5-aeebbf26d5a4"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:58.932164 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:58.932116    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-data-science-collector-ta-client-cert" pod "5239e678-35b6-4e45-bee5-aeebbf26d5a4" (UID: "5239e678-35b6-4e45-bee5-aeebbf26d5a4"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:59.030695 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.030669    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5239e678-35b6-4e45-bee5-aeebbf26d5a4-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.030695 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.030691    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.030818 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.030701    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x8j7\" (UniqueName: \"kubernetes.io/projected/5239e678-35b6-4e45-bee5-aeebbf26d5a4-kube-api-access-5x8j7\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.030818 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.030710    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.030818 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.030718    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5239e678-35b6-4e45-bee5-aeebbf26d5a4-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.101621 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.101590    4849 generic.go:356] "Generic (PLEG): container finished" podID="5239e678-35b6-4e45-bee5-aeebbf26d5a4" containerID="1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212" exitCode=0
Oct 30 10:56:59.101722 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.101643    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.101722 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.101679    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5239e678-35b6-4e45-bee5-aeebbf26d5a4","Type":"ContainerDied","Data":"1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212"}
Oct 30 10:56:59.101836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.101738    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5239e678-35b6-4e45-bee5-aeebbf26d5a4","Type":"ContainerDied","Data":"a2c8657ba93f248bec4ce0ce8481f711a6416cb98eb3448bd216379337a51d1d"}
Oct 30 10:56:59.101836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.101770    4849 scope.go:122] "RemoveContainer" containerID="1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212"
Oct 30 10:56:59.104065 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104043    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerStarted","Data":"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"}
Oct 30 10:56:59.104145 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104094    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerStarted","Data":"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"}
Oct 30 10:56:59.104145 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104124    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerStarted","Data":"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"}
Oct 30 10:56:59.104145 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104140    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerStarted","Data":"8d8470922c19fb368288d0cb7135ba82673099dd1516a8559422fd93e266879b"}
Oct 30 10:56:59.104266 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104188    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo" containerID="cri-o://122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202" gracePeriod=30
Oct 30 10:56:59.104266 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104223    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway-opa" containerID="cri-o://e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83" gracePeriod=30
Oct 30 10:56:59.104345 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104316    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway" containerID="cri-o://a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b" gracePeriod=30
Oct 30 10:56:59.104423 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104406    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:59.104512 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.104446    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:59.107841 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.107822    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:59.110618 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.110592    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 10:56:59.115877 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.115857    4849 scope.go:122] "RemoveContainer" containerID="1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212"
Oct 30 10:56:59.116174 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:56:59.116154    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212\": container with ID starting with 1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212 not found: ID does not exist" containerID="1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212"
Oct 30 10:56:59.116252 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.116185    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212"} err="failed to get container status \"1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212\": rpc error: code = NotFound desc = could not find container \"1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212\": container with ID starting with 1d150e207795a27dcafcae40f6eecf45f167a2196b4a3570b03a73c4ab2f4212 not found: ID does not exist"
Oct 30 10:56:59.186666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.186587    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podStartSLOduration=10.186575491 podStartE2EDuration="10.186575491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:56:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:56:59.186054284 +0000 UTC m=+716.566503725" watchObservedRunningTime="2026-10-30 10:56:59.186575491 +0000 UTC m=+716.567024930"
Oct 30 10:56:59.225740 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.225719    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:59.230339 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.230318    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:59.266564 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.266543    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:59.266964 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.266948    4849 cpu_manager.go:414] "containerMap: removing container" podUID="5239e678-35b6-4e45-bee5-aeebbf26d5a4" containerName="otc-container"
Oct 30 10:56:59.267080 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.266965    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5239e678-35b6-4e45-bee5-aeebbf26d5a4" containerName="otc-container"
Oct 30 10:56:59.267080 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.267053    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="5239e678-35b6-4e45-bee5-aeebbf26d5a4" containerName="otc-container"
Oct 30 10:56:59.270072 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.270054    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.289604 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.289583    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:56:59.289750 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.289701    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:56:59.289817 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.289797    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:56:59.289891 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.289876    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:56:59.290075 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.289975    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:56:59.291386 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.291287    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:59.333644 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.333618    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.333733 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.333654    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h98j\" (UniqueName: \"kubernetes.io/projected/d0e80763-bb91-425c-b91d-577ecd2acd93-kube-api-access-9h98j\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.333733 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.333687    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.333813 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.333791    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.333852 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.333844    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0e80763-bb91-425c-b91d-577ecd2acd93-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.434341 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.434311    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0e80763-bb91-425c-b91d-577ecd2acd93-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.434483 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.434398    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.434483 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.434440    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h98j\" (UniqueName: \"kubernetes.io/projected/d0e80763-bb91-425c-b91d-577ecd2acd93-kube-api-access-9h98j\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.434606 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.434494    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.434606 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.434527    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.435172 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.435143    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0e80763-bb91-425c-b91d-577ecd2acd93-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.437015 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.436964    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.437140 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.437063    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.437282 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.437259    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.449519 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.449501    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:56:59.461038 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.461004    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h98j\" (UniqueName: \"kubernetes.io/projected/d0e80763-bb91-425c-b91d-577ecd2acd93-kube-api-access-9h98j\") pod \"data-science-collector-collector-0\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.535822 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.535797    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-conf\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.535960 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.535941    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.536061 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.535990    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.536121 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536057    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-ca-bundle\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536192    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-conf" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536302    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-gateway-rbac\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536348    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536404    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-gateway-tenants\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-gateway-tenants\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536427    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/625b195e-5962-4216-aaf5-a5f257910389-kube-api-access-2zv7v\" (UniqueName: \"kubernetes.io/projected/625b195e-5962-4216-aaf5-a5f257910389-kube-api-access-2zv7v\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536399    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-serving-cabundle" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-serving-cabundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536439    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-ca-bundle" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:56:59.536525 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536488    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-mtls\") pod \"625b195e-5962-4216-aaf5-a5f257910389\" (UID: \"625b195e-5962-4216-aaf5-a5f257910389\") "
Oct 30 10:56:59.537280 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536725    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.537280 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536737    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-serving-cabundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.537280 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536747    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.537280 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.536749    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-gateway-rbac" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "gateway-rbac". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:56:59.538784 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.538762    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-gateway-serving-cert" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-gateway-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:59.538925 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.538907    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625b195e-5962-4216-aaf5-a5f257910389-kube-api-access-2zv7v" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "kube-api-access-2zv7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:56:59.539058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.539003    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-mtls" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:59.539143 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.539061    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 10:56:59.539143 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.539069    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-gateway-tenants" pod "625b195e-5962-4216-aaf5-a5f257910389" (UID: "625b195e-5962-4216-aaf5-a5f257910389"). InnerVolumeSpecName "gateway-tenants". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:56:59.583788 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.583771    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:56:59.639936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.637571    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 10:56:59.639936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.639860    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/625b195e-5962-4216-aaf5-a5f257910389-gateway-rbac\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.639936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.639882    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-gateway-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.639936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.639892    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-gateway-tenants\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.639936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.639901    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zv7v\" (UniqueName: \"kubernetes.io/projected/625b195e-5962-4216-aaf5-a5f257910389-kube-api-access-2zv7v\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.639936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.639917    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/625b195e-5962-4216-aaf5-a5f257910389-tempo-data-science-tempomonolithic-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:56:59.652587 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.652570    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e") on node "ip-10-0-140-230.ec2.internal"
Oct 30 10:56:59.740898 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.740846    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\""
Oct 30 10:56:59.884827 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.884805    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:56:59.886624 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:56:59.886599    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e80763_bb91_425c_b91d_577ecd2acd93.slice/crio-6e9811c7de33ecdf683c1aeb3b2089a98e68b7e0040ba6fcb624b31514389ba4 WatchSource:0}: Error finding container 6e9811c7de33ecdf683c1aeb3b2089a98e68b7e0040ba6fcb624b31514389ba4: Status 404 returned error can't find the container with id 6e9811c7de33ecdf683c1aeb3b2089a98e68b7e0040ba6fcb624b31514389ba4
Oct 30 10:56:59.888755 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.888726    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:56:59.888843 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:56:59.888787    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:00.112681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112651    4849 generic.go:356] "Generic (PLEG): container finished" podID="625b195e-5962-4216-aaf5-a5f257910389" containerID="e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83" exitCode=0
Oct 30 10:57:00.112681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112679    4849 generic.go:356] "Generic (PLEG): container finished" podID="625b195e-5962-4216-aaf5-a5f257910389" containerID="a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b" exitCode=0
Oct 30 10:57:00.112973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112688    4849 generic.go:356] "Generic (PLEG): container finished" podID="625b195e-5962-4216-aaf5-a5f257910389" containerID="122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202" exitCode=0
Oct 30 10:57:00.112973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112712    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:00.112973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112728    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerDied","Data":"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"}
Oct 30 10:57:00.112973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112781    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerDied","Data":"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"}
Oct 30 10:57:00.112973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112807    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerDied","Data":"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"}
Oct 30 10:57:00.112973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112830    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"625b195e-5962-4216-aaf5-a5f257910389","Type":"ContainerDied","Data":"8d8470922c19fb368288d0cb7135ba82673099dd1516a8559422fd93e266879b"}
Oct 30 10:57:00.112973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.112865    4849 scope.go:122] "RemoveContainer" containerID="e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"
Oct 30 10:57:00.114511 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.114477    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0e80763-bb91-425c-b91d-577ecd2acd93","Type":"ContainerStarted","Data":"986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b"}
Oct 30 10:57:00.114614 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.114516    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0e80763-bb91-425c-b91d-577ecd2acd93","Type":"ContainerStarted","Data":"6e9811c7de33ecdf683c1aeb3b2089a98e68b7e0040ba6fcb624b31514389ba4"}
Oct 30 10:57:00.122475 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.122457    4849 scope.go:122] "RemoveContainer" containerID="a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"
Oct 30 10:57:00.132877 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.132859    4849 scope.go:122] "RemoveContainer" containerID="122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"
Oct 30 10:57:00.141133 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.141102    4849 scope.go:122] "RemoveContainer" containerID="e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"
Oct 30 10:57:00.141449 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:00.141426    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83\": container with ID starting with e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83 not found: ID does not exist" containerID="e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"
Oct 30 10:57:00.141511 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.141455    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"} err="failed to get container status \"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83\": rpc error: code = NotFound desc = could not find container \"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83\": container with ID starting with e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83 not found: ID does not exist"
Oct 30 10:57:00.141511 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.141473    4849 scope.go:122] "RemoveContainer" containerID="a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"
Oct 30 10:57:00.141722 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:00.141709    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b\": container with ID starting with a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b not found: ID does not exist" containerID="a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"
Oct 30 10:57:00.141762 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.141727    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"} err="failed to get container status \"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b\": rpc error: code = NotFound desc = could not find container \"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b\": container with ID starting with a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b not found: ID does not exist"
Oct 30 10:57:00.141762 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.141738    4849 scope.go:122] "RemoveContainer" containerID="122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"
Oct 30 10:57:00.141952 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:00.141936    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202\": container with ID starting with 122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202 not found: ID does not exist" containerID="122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"
Oct 30 10:57:00.141998 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.141954    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"} err="failed to get container status \"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202\": rpc error: code = NotFound desc = could not find container \"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202\": container with ID starting with 122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202 not found: ID does not exist"
Oct 30 10:57:00.141998 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.141964    4849 scope.go:122] "RemoveContainer" containerID="e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"
Oct 30 10:57:00.142278 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142256    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"} err="failed to get container status \"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83\": rpc error: code = NotFound desc = could not find container \"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83\": container with ID starting with e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83 not found: ID does not exist"
Oct 30 10:57:00.142327 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142279    4849 scope.go:122] "RemoveContainer" containerID="a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"
Oct 30 10:57:00.142526 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142504    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"} err="failed to get container status \"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b\": rpc error: code = NotFound desc = could not find container \"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b\": container with ID starting with a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b not found: ID does not exist"
Oct 30 10:57:00.142596 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142528    4849 scope.go:122] "RemoveContainer" containerID="122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"
Oct 30 10:57:00.142783 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142762    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"} err="failed to get container status \"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202\": rpc error: code = NotFound desc = could not find container \"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202\": container with ID starting with 122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202 not found: ID does not exist"
Oct 30 10:57:00.142833 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142783    4849 scope.go:122] "RemoveContainer" containerID="e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"
Oct 30 10:57:00.142985 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142965    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83"} err="failed to get container status \"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83\": rpc error: code = NotFound desc = could not find container \"e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83\": container with ID starting with e0e5b1aa573511c3a19ef934dcac060cf85ec9a4d6f576c5ce26eabac8fa2f83 not found: ID does not exist"
Oct 30 10:57:00.143078 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.142987    4849 scope.go:122] "RemoveContainer" containerID="a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"
Oct 30 10:57:00.143200 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.143182    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b"} err="failed to get container status \"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b\": rpc error: code = NotFound desc = could not find container \"a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b\": container with ID starting with a67866b36f54189de38a2d9ecc92542025378f0d447395445aa9c4a9cd22b19b not found: ID does not exist"
Oct 30 10:57:00.143275 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.143202    4849 scope.go:122] "RemoveContainer" containerID="122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"
Oct 30 10:57:00.143399 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.143374    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202"} err="failed to get container status \"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202\": rpc error: code = NotFound desc = could not find container \"122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202\": container with ID starting with 122c90f10912435f0654c8c8f6e943b7d5282bcf148d3b896f933890f4d34202 not found: ID does not exist"
Oct 30 10:57:00.171550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.171527    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:00.184877 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.184848    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:00.219860 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.219823    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.219811186 podStartE2EDuration="1.219811186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:00.217915969 +0000 UTC m=+717.598365430" watchObservedRunningTime="2026-10-30 10:57:00.219811186 +0000 UTC m=+717.600260624"
Oct 30 10:57:00.754545 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.754513    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5239e678-35b6-4e45-bee5-aeebbf26d5a4" path="/var/lib/kubelet/pods/5239e678-35b6-4e45-bee5-aeebbf26d5a4/volumes"
Oct 30 10:57:00.754924 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:00.754911    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="625b195e-5962-4216-aaf5-a5f257910389" path="/var/lib/kubelet/pods/625b195e-5962-4216-aaf5-a5f257910389/volumes"
Oct 30 10:57:07.354915 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.354877    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:07.355720 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.355699    4849 cpu_manager.go:414] "containerMap: removing container" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway-opa"
Oct 30 10:57:07.355766 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.355723    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway-opa"
Oct 30 10:57:07.355766 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.355751    4849 cpu_manager.go:414] "containerMap: removing container" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway"
Oct 30 10:57:07.355766 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.355759    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway"
Oct 30 10:57:07.355857 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.355803    4849 cpu_manager.go:414] "containerMap: removing container" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo"
Oct 30 10:57:07.355857 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.355810    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo"
Oct 30 10:57:07.355997 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.355985    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo"
Oct 30 10:57:07.356057 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.356012    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway-opa"
Oct 30 10:57:07.356057 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.356039    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="625b195e-5962-4216-aaf5-a5f257910389" containerName="tempo-gateway"
Oct 30 10:57:07.362475 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.362451    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.381619 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.381587    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:57:07.384007 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.383982    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:07.395517 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.395494    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway-serving-cert\""
Oct 30 10:57:07.395615 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.395513    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-config\""
Oct 30 10:57:07.395615 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.395524    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-serving-cabundle\""
Oct 30 10:57:07.395615 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.395525    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:57:07.395615 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.395579    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-ca-bundle\""
Oct 30 10:57:07.482221 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.482198    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-mtls\""
Oct 30 10:57:07.482506 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.482488    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-dockercfg-w655h\""
Oct 30 10:57:07.503849 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.503826    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpcnz\" (UniqueName: \"kubernetes.io/projected/7e556594-b999-49f8-84ab-8d65f06612b0-kube-api-access-cpcnz\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.503909 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.503859    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.503909 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.503885    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.503975 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.503939    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.504059 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.504042    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.504093 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.504070    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.504168 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.504152    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.504204 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.504183    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605533 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605468    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605533 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605504    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605697 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605534    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605697 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605557    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605697 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605593    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605697 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605624    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605889 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605732    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpcnz\" (UniqueName: \"kubernetes.io/projected/7e556594-b999-49f8-84ab-8d65f06612b0-kube-api-access-cpcnz\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.605889 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.605769    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.606375 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.606353    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.606528 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.606503    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.606651 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.606367    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.606651 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.606569    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.608351 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.608327    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.608437 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.608377    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.608437 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.608404    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.631913 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.631883    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpcnz\" (UniqueName: \"kubernetes.io/projected/7e556594-b999-49f8-84ab-8d65f06612b0-kube-api-access-cpcnz\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.807310 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.807281    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.840819 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.840794    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") device path: \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.907994 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.907940    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.908109 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.908056    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") DevicePath \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:07.923560 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:07.923540    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:08.026267 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:08.026245    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/fd3ed81014d76a975b5ba6f044c00684e55bfb3daddf6ff0e225ceff74c57792/globalmount\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:08.054166 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:08.054144    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:08.278047 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:08.277965    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:08.612047 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:08.611990    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:08.630775 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:08.630747    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:08.631218 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:08.630980    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="d0e80763-bb91-425c-b91d-577ecd2acd93" containerName="otc-container" containerID="cri-o://986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b" gracePeriod=30
Oct 30 10:57:08.635079 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:08.635046    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e556594_b999_49f8_84ab_8d65f06612b0.slice/crio-64c23de32ab0a6f9d217de4c8a5aa3c7a050debe4568dc84832c342750b67d3d WatchSource:0}: Error finding container 64c23de32ab0a6f9d217de4c8a5aa3c7a050debe4568dc84832c342750b67d3d: Status 404 returned error can't find the container with id 64c23de32ab0a6f9d217de4c8a5aa3c7a050debe4568dc84832c342750b67d3d
Oct 30 10:57:08.912796 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:08.912773    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.017766 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.017719    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0e80763-bb91-425c-b91d-577ecd2acd93-kube-api-access-9h98j\" (UniqueName: \"kubernetes.io/projected/d0e80763-bb91-425c-b91d-577ecd2acd93-kube-api-access-9h98j\") pod \"d0e80763-bb91-425c-b91d-577ecd2acd93\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") "
Oct 30 10:57:09.017936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.017808    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0e80763-bb91-425c-b91d-577ecd2acd93-otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0e80763-bb91-425c-b91d-577ecd2acd93-otc-internal\") pod \"d0e80763-bb91-425c-b91d-577ecd2acd93\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") "
Oct 30 10:57:09.017936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.017919    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-data-science-collector-ta-client-cert\") pod \"d0e80763-bb91-425c-b91d-577ecd2acd93\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") "
Oct 30 10:57:09.018065 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.017981    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-tls-certs\") pod \"d0e80763-bb91-425c-b91d-577ecd2acd93\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") "
Oct 30 10:57:09.018134 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.018061    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-monitoring-tls-certs\") pod \"d0e80763-bb91-425c-b91d-577ecd2acd93\" (UID: \"d0e80763-bb91-425c-b91d-577ecd2acd93\") "
Oct 30 10:57:09.018253 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.018209    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e80763-bb91-425c-b91d-577ecd2acd93-otc-internal" pod "d0e80763-bb91-425c-b91d-577ecd2acd93" (UID: "d0e80763-bb91-425c-b91d-577ecd2acd93"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:09.018425 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.018404    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d0e80763-bb91-425c-b91d-577ecd2acd93-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:09.019852 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.019830    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e80763-bb91-425c-b91d-577ecd2acd93-kube-api-access-9h98j" pod "d0e80763-bb91-425c-b91d-577ecd2acd93" (UID: "d0e80763-bb91-425c-b91d-577ecd2acd93"). InnerVolumeSpecName "kube-api-access-9h98j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:09.020069 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.020051    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-tls-certs" pod "d0e80763-bb91-425c-b91d-577ecd2acd93" (UID: "d0e80763-bb91-425c-b91d-577ecd2acd93"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:09.020397 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.020380    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-data-science-collector-ta-client-cert" pod "d0e80763-bb91-425c-b91d-577ecd2acd93" (UID: "d0e80763-bb91-425c-b91d-577ecd2acd93"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:09.020519 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.020493    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-monitoring-tls-certs" pod "d0e80763-bb91-425c-b91d-577ecd2acd93" (UID: "d0e80763-bb91-425c-b91d-577ecd2acd93"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:09.119868 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.119815    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:09.119868 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.119837    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h98j\" (UniqueName: \"kubernetes.io/projected/d0e80763-bb91-425c-b91d-577ecd2acd93-kube-api-access-9h98j\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:09.119868 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.119848    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:09.119868 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.119859    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e80763-bb91-425c-b91d-577ecd2acd93-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:09.162706 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.162682    4849 generic.go:356] "Generic (PLEG): container finished" podID="d0e80763-bb91-425c-b91d-577ecd2acd93" containerID="986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b" exitCode=0
Oct 30 10:57:09.162795 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.162735    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.162795 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.162768    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0e80763-bb91-425c-b91d-577ecd2acd93","Type":"ContainerDied","Data":"986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b"}
Oct 30 10:57:09.162872 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.162819    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d0e80763-bb91-425c-b91d-577ecd2acd93","Type":"ContainerDied","Data":"6e9811c7de33ecdf683c1aeb3b2089a98e68b7e0040ba6fcb624b31514389ba4"}
Oct 30 10:57:09.162872 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.162842    4849 scope.go:122] "RemoveContainer" containerID="986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b"
Oct 30 10:57:09.165329 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.165301    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerStarted","Data":"a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b"}
Oct 30 10:57:09.165427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.165370    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerStarted","Data":"bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec"}
Oct 30 10:57:09.165427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.165395    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerStarted","Data":"48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878"}
Oct 30 10:57:09.165427 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.165412    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerStarted","Data":"64c23de32ab0a6f9d217de4c8a5aa3c7a050debe4568dc84832c342750b67d3d"}
Oct 30 10:57:09.165541 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.165432    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:09.165541 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.165457    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:09.168277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.168224    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:09.173015 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.173000    4849 scope.go:122] "RemoveContainer" containerID="986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b"
Oct 30 10:57:09.173382 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:09.173362    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b\": container with ID starting with 986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b not found: ID does not exist" containerID="986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b"
Oct 30 10:57:09.173441 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.173391    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b"} err="failed to get container status \"986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b\": rpc error: code = NotFound desc = could not find container \"986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b\": container with ID starting with 986a6bedde12a1078325f92b08f52557c6a8edca6d873afb7a3f6d930fe3f85b not found: ID does not exist"
Oct 30 10:57:09.174592 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.174576    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:09.217679 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.217657    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:09.221167 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.221147    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:09.258687 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.258665    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:09.259237 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.259219    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d0e80763-bb91-425c-b91d-577ecd2acd93" containerName="otc-container"
Oct 30 10:57:09.259285 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.259241    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0e80763-bb91-425c-b91d-577ecd2acd93" containerName="otc-container"
Oct 30 10:57:09.259367 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.259355    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0e80763-bb91-425c-b91d-577ecd2acd93" containerName="otc-container"
Oct 30 10:57:09.265836 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.265811    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.271643 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.271595    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podStartSLOduration=2.271580108 podStartE2EDuration="2.271580108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 10:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:09.270390075 +0000 UTC m=+726.650839553" watchObservedRunningTime="2026-10-30 10:57:09.271580108 +0000 UTC m=+726.652029548"
Oct 30 10:57:09.282742 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.282712    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:57:09.282855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.282793    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:57:09.282855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.282799    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:57:09.282855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.282842    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:57:09.283529 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.283510    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:09.283877 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.283860    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-7c92b5b1\""
Oct 30 10:57:09.423262 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.423183    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxql6\" (UniqueName: \"kubernetes.io/projected/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-kube-api-access-jxql6\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.423262 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.423235    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.423445 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.423337    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.423445 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.423380    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.423445 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.423417    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.430995 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:09.430966    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:57:09.524506 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.524480    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.524598 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.524514    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.524598 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.524550    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxql6\" (UniqueName: \"kubernetes.io/projected/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-kube-api-access-jxql6\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.524598 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.524592    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.524748 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.524641    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.525169 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.525143    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.526895 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.526874    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.526969 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.526917    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.527034 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.526994    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.548706 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.548683    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxql6\" (UniqueName: \"kubernetes.io/projected/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-kube-api-access-jxql6\") pod \"data-science-collector-collector-0\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.583651 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.583631    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:09.884110 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.884087    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:09.886017 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:09.885991    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e96c67c_524b_4e6e_8edb_f3a416cd1fdf.slice/crio-b8da1d4680c316b3a31754ef1a3fa42967b04e6f9388f14fd5f9bf2a6a6f6ce2 WatchSource:0}: Error finding container b8da1d4680c316b3a31754ef1a3fa42967b04e6f9388f14fd5f9bf2a6a6f6ce2: Status 404 returned error can't find the container with id b8da1d4680c316b3a31754ef1a3fa42967b04e6f9388f14fd5f9bf2a6a6f6ce2
Oct 30 10:57:09.888208 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.888173    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:09.888312 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:09.888243    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:10.177199 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:10.177113    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf","Type":"ContainerStarted","Data":"9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89"}
Oct 30 10:57:10.177199 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:10.177170    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf","Type":"ContainerStarted","Data":"b8da1d4680c316b3a31754ef1a3fa42967b04e6f9388f14fd5f9bf2a6a6f6ce2"}
Oct 30 10:57:10.177413 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:10.177256    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:10.224812 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:10.224764    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.224751418 podStartE2EDuration="1.224751418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:57:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:10.222734167 +0000 UTC m=+727.603183633" watchObservedRunningTime="2026-10-30 10:57:10.224751418 +0000 UTC m=+727.605200901"
Oct 30 10:57:10.753849 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:10.753816    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0e80763-bb91-425c-b91d-577ecd2acd93" path="/var/lib/kubelet/pods/d0e80763-bb91-425c-b91d-577ecd2acd93/volumes"
Oct 30 10:57:15.493304 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:15.493265    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:15.493781 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:15.493528    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo" containerID="cri-o://48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878" gracePeriod=30
Oct 30 10:57:15.493781 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:15.493553    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway-opa" containerID="cri-o://a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b" gracePeriod=30
Oct 30 10:57:15.493781 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:15.493661    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway" containerID="cri-o://bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec" gracePeriod=30
Oct 30 10:57:16.209417 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.209384    4849 generic.go:356] "Generic (PLEG): container finished" podID="7e556594-b999-49f8-84ab-8d65f06612b0" containerID="a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b" exitCode=0
Oct 30 10:57:16.209417 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.209411    4849 generic.go:356] "Generic (PLEG): container finished" podID="7e556594-b999-49f8-84ab-8d65f06612b0" containerID="48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878" exitCode=0
Oct 30 10:57:16.209607 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.209446    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerDied","Data":"a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b"}
Oct 30 10:57:16.209607 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.209485    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerDied","Data":"48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878"}
Oct 30 10:57:16.677001 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.676963    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:16.677435 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.677262    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" containerName="otc-container" containerID="cri-o://9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89" gracePeriod=30
Oct 30 10:57:16.805857 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.805814    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:16.885953 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.885926    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-conf\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.886121 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.885974    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-mtls\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.886121 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886014    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.886121 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886049    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-rbac\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.886121 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886072    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e556594-b999-49f8-84ab-8d65f06612b0-kube-api-access-cpcnz\" (UniqueName: \"kubernetes.io/projected/7e556594-b999-49f8-84ab-8d65f06612b0-kube-api-access-cpcnz\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.886121 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886096    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.886361 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886191    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.886424 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886373    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-conf" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:16.886480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886433    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-serving-cabundle" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-serving-cabundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:16.886480 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886463    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:16.886729 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.886709    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-rbac" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "gateway-rbac". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:16.888938 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.888913    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e556594-b999-49f8-84ab-8d65f06612b0-kube-api-access-cpcnz" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "kube-api-access-cpcnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:16.889085 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.888961    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-gateway-serving-cert" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-gateway-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:16.889085 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.889014    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-mtls" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:16.889190 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.889171    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 10:57:16.911426 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.911406    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:16.986899 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.986869    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-data-science-collector-ta-client-cert\") pod \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") "
Oct 30 10:57:16.987058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.986926    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-kube-api-access-jxql6\" (UniqueName: \"kubernetes.io/projected/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-kube-api-access-jxql6\") pod \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") "
Oct 30 10:57:16.987058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.986945    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-otc-internal\" (UniqueName: \"kubernetes.io/configmap/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-otc-internal\") pod \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") "
Oct 30 10:57:16.987058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.986972    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-tls-certs\") pod \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") "
Oct 30 10:57:16.987058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987003    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.987058 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987054    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-monitoring-tls-certs\") pod \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\" (UID: \"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf\") "
Oct 30 10:57:16.987358 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987097    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-tenants\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-tenants\") pod \"7e556594-b999-49f8-84ab-8d65f06612b0\" (UID: \"7e556594-b999-49f8-84ab-8d65f06612b0\") "
Oct 30 10:57:16.987416 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987352    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-otc-internal" pod "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" (UID: "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:16.987473 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987417    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 10:57:16.987473 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987417    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-ca-bundle" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:16.987473 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987439    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:16.987473 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987456    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-serving-cabundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:16.987473 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987470    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-rbac\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:16.987737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987484    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpcnz\" (UniqueName: \"kubernetes.io/projected/7e556594-b999-49f8-84ab-8d65f06612b0-kube-api-access-cpcnz\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:16.987737 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.987498    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-gateway-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:16.989769 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.989737    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-tls-certs" pod "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" (UID: "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:16.989879 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.989775    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-kube-api-access-jxql6" pod "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" (UID: "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf"). InnerVolumeSpecName "kube-api-access-jxql6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:16.989879 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.989774    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-monitoring-tls-certs" pod "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" (UID: "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:16.989879 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.989804    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-data-science-collector-ta-client-cert" pod "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" (UID: "4e96c67c-524b-4e6e-8edb-f3a416cd1fdf"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:16.990140 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.990124    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-tenants" pod "7e556594-b999-49f8-84ab-8d65f06612b0" (UID: "7e556594-b999-49f8-84ab-8d65f06612b0"). InnerVolumeSpecName "gateway-tenants". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:16.998286 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:16.998273    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e") on node "ip-10-0-140-230.ec2.internal"
Oct 30 10:57:17.088255 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088233    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxql6\" (UniqueName: \"kubernetes.io/projected/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-kube-api-access-jxql6\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:17.088255 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088251    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:17.088377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088262    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:17.088377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088270    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\""
Oct 30 10:57:17.088377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088280    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e556594-b999-49f8-84ab-8d65f06612b0-tempo-data-science-tempomonolithic-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:17.088377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088289    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:17.088377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088298    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/7e556594-b999-49f8-84ab-8d65f06612b0-gateway-tenants\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:17.088377 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.088306    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:17.216883 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.216822    4849 generic.go:356] "Generic (PLEG): container finished" podID="4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" containerID="9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89" exitCode=0
Oct 30 10:57:17.216883 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.216876    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.217060 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.216907    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf","Type":"ContainerDied","Data":"9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89"}
Oct 30 10:57:17.217060 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.216958    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4e96c67c-524b-4e6e-8edb-f3a416cd1fdf","Type":"ContainerDied","Data":"b8da1d4680c316b3a31754ef1a3fa42967b04e6f9388f14fd5f9bf2a6a6f6ce2"}
Oct 30 10:57:17.217060 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.216980    4849 scope.go:122] "RemoveContainer" containerID="9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89"
Oct 30 10:57:17.219103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.219079    4849 generic.go:356] "Generic (PLEG): container finished" podID="7e556594-b999-49f8-84ab-8d65f06612b0" containerID="bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec" exitCode=0
Oct 30 10:57:17.219186 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.219123    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerDied","Data":"bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec"}
Oct 30 10:57:17.219186 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.219133    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:17.219186 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.219170    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"7e556594-b999-49f8-84ab-8d65f06612b0","Type":"ContainerDied","Data":"64c23de32ab0a6f9d217de4c8a5aa3c7a050debe4568dc84832c342750b67d3d"}
Oct 30 10:57:17.228057 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.228007    4849 scope.go:122] "RemoveContainer" containerID="9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89"
Oct 30 10:57:17.228309 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:17.228293    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89\": container with ID starting with 9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89 not found: ID does not exist" containerID="9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89"
Oct 30 10:57:17.228376 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.228318    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89"} err="failed to get container status \"9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89\": rpc error: code = NotFound desc = could not find container \"9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89\": container with ID starting with 9847dc1d17bdc653cb3c4c174c889692e950030a8262e8b76ae45c19e8f44a89 not found: ID does not exist"
Oct 30 10:57:17.228376 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.228343    4849 scope.go:122] "RemoveContainer" containerID="a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b"
Oct 30 10:57:17.236168 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.236062    4849 scope.go:122] "RemoveContainer" containerID="bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec"
Oct 30 10:57:17.244350 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.244335    4849 scope.go:122] "RemoveContainer" containerID="48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878"
Oct 30 10:57:17.252247 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.252217    4849 scope.go:122] "RemoveContainer" containerID="a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b"
Oct 30 10:57:17.252497 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:17.252480    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b\": container with ID starting with a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b not found: ID does not exist" containerID="a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b"
Oct 30 10:57:17.252546 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.252508    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b"} err="failed to get container status \"a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b\": rpc error: code = NotFound desc = could not find container \"a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b\": container with ID starting with a87e2291b56f4c39ce9ff919dd403cf2052d7ae388d4a15c4a02173b56f2391b not found: ID does not exist"
Oct 30 10:57:17.252546 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.252526    4849 scope.go:122] "RemoveContainer" containerID="bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec"
Oct 30 10:57:17.252790 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:17.252772    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec\": container with ID starting with bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec not found: ID does not exist" containerID="bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec"
Oct 30 10:57:17.252848 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.252796    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec"} err="failed to get container status \"bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec\": rpc error: code = NotFound desc = could not find container \"bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec\": container with ID starting with bcdfcc52d0bb104994d340e185e9ff650c68ddea62451dc3b89776a81da020ec not found: ID does not exist"
Oct 30 10:57:17.252848 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.252809    4849 scope.go:122] "RemoveContainer" containerID="48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878"
Oct 30 10:57:17.253063 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:17.253046    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878\": container with ID starting with 48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878 not found: ID does not exist" containerID="48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878"
Oct 30 10:57:17.253138 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.253066    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878"} err="failed to get container status \"48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878\": rpc error: code = NotFound desc = could not find container \"48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878\": container with ID starting with 48d67a953f5174a3cd5c42c0eee879ff7973c98a144a9daebcc677654785e878 not found: ID does not exist"
Oct 30 10:57:17.273260 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.273239    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:17.277108 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.277090    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:17.314379 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.314356    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:17.316932 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.316914    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:17.353293 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353268    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:17.353677 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353660    4849 cpu_manager.go:414] "containerMap: removing container" podUID="4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" containerName="otc-container"
Oct 30 10:57:17.353758 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353678    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" containerName="otc-container"
Oct 30 10:57:17.353758 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353735    4849 cpu_manager.go:414] "containerMap: removing container" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo"
Oct 30 10:57:17.353758 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353744    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo"
Oct 30 10:57:17.353758 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353756    4849 cpu_manager.go:414] "containerMap: removing container" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway-opa"
Oct 30 10:57:17.353962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353763    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway-opa"
Oct 30 10:57:17.353962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353779    4849 cpu_manager.go:414] "containerMap: removing container" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway"
Oct 30 10:57:17.353962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353788    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway"
Oct 30 10:57:17.353962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353883    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway"
Oct 30 10:57:17.353962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353899    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" containerName="otc-container"
Oct 30 10:57:17.353962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353916    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo"
Oct 30 10:57:17.353962 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.353926    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" containerName="tempo-gateway-opa"
Oct 30 10:57:17.358609 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.358592    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.376183 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.376162    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:57:17.376805 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.376754    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:57:17.376912 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.376844    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:57:17.376912 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.376892    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:57:17.377054 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.376931    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:57:17.378405 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.378384    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:17.391672 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.391651    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c833a520-8403-49f3-87cc-7076a508e8b5-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.391781 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.391679    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.391781 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.391762    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfdd2\" (UniqueName: \"kubernetes.io/projected/c833a520-8403-49f3-87cc-7076a508e8b5-kube-api-access-dfdd2\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.391867 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.391800    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.391867 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.391818    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.492696 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.492633    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdd2\" (UniqueName: \"kubernetes.io/projected/c833a520-8403-49f3-87cc-7076a508e8b5-kube-api-access-dfdd2\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.492696 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.492690    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.492853 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.492719    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.492853 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.492785    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c833a520-8403-49f3-87cc-7076a508e8b5-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.492853 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.492809    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.493657 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.493629    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c833a520-8403-49f3-87cc-7076a508e8b5-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.495262 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.495237    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.495262 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.495247    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.495374 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.495285    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.516687 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.516664    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfdd2\" (UniqueName: \"kubernetes.io/projected/c833a520-8403-49f3-87cc-7076a508e8b5-kube-api-access-dfdd2\") pod \"data-science-collector-collector-0\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.671311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.671290    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:17.968176 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.968156    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:17.970157 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:17.970131    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc833a520_8403_49f3_87cc_7076a508e8b5.slice/crio-da6772416f948b8ba64c400314943b6ccd51d2267c9eae26050dfc103fc71500 WatchSource:0}: Error finding container da6772416f948b8ba64c400314943b6ccd51d2267c9eae26050dfc103fc71500: Status 404 returned error can't find the container with id da6772416f948b8ba64c400314943b6ccd51d2267c9eae26050dfc103fc71500
Oct 30 10:57:17.972252 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.972223    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:17.972353 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:17.972293    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:18.225845 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:18.225748    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c833a520-8403-49f3-87cc-7076a508e8b5","Type":"ContainerStarted","Data":"b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a"}
Oct 30 10:57:18.225845 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:18.225804    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c833a520-8403-49f3-87cc-7076a508e8b5","Type":"ContainerStarted","Data":"da6772416f948b8ba64c400314943b6ccd51d2267c9eae26050dfc103fc71500"}
Oct 30 10:57:18.270942 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:18.270894    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.27087882 podStartE2EDuration="1.27087882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:18.269258015 +0000 UTC m=+735.649707453" watchObservedRunningTime="2026-10-30 10:57:18.27087882 +0000 UTC m=+735.651328260"
Oct 30 10:57:18.753868 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:18.753835    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e96c67c-524b-4e6e-8edb-f3a416cd1fdf" path="/var/lib/kubelet/pods/4e96c67c-524b-4e6e-8edb-f3a416cd1fdf/volumes"
Oct 30 10:57:18.754314 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:18.754298    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e556594-b999-49f8-84ab-8d65f06612b0" path="/var/lib/kubelet/pods/7e556594-b999-49f8-84ab-8d65f06612b0/volumes"
Oct 30 10:57:25.720297 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:25.720267    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:25.720756 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:25.720498    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="c833a520-8403-49f3-87cc-7076a508e8b5" containerName="otc-container" containerID="cri-o://b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a" gracePeriod=30
Oct 30 10:57:25.970883 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:25.970832    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.060755 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.060731    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-tls-certs\") pod \"c833a520-8403-49f3-87cc-7076a508e8b5\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") "
Oct 30 10:57:26.060896 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.060810    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c833a520-8403-49f3-87cc-7076a508e8b5-kube-api-access-dfdd2\" (UniqueName: \"kubernetes.io/projected/c833a520-8403-49f3-87cc-7076a508e8b5-kube-api-access-dfdd2\") pod \"c833a520-8403-49f3-87cc-7076a508e8b5\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") "
Oct 30 10:57:26.060896 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.060837    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-data-science-collector-ta-client-cert\") pod \"c833a520-8403-49f3-87cc-7076a508e8b5\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") "
Oct 30 10:57:26.060896 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.060879    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c833a520-8403-49f3-87cc-7076a508e8b5-otc-internal\" (UniqueName: \"kubernetes.io/configmap/c833a520-8403-49f3-87cc-7076a508e8b5-otc-internal\") pod \"c833a520-8403-49f3-87cc-7076a508e8b5\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") "
Oct 30 10:57:26.061103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.060899    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-monitoring-tls-certs\") pod \"c833a520-8403-49f3-87cc-7076a508e8b5\" (UID: \"c833a520-8403-49f3-87cc-7076a508e8b5\") "
Oct 30 10:57:26.061328 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.061292    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c833a520-8403-49f3-87cc-7076a508e8b5-otc-internal" pod "c833a520-8403-49f3-87cc-7076a508e8b5" (UID: "c833a520-8403-49f3-87cc-7076a508e8b5"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:26.062807 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.062777    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-tls-certs" pod "c833a520-8403-49f3-87cc-7076a508e8b5" (UID: "c833a520-8403-49f3-87cc-7076a508e8b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:26.062892 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.062868    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-data-science-collector-ta-client-cert" pod "c833a520-8403-49f3-87cc-7076a508e8b5" (UID: "c833a520-8403-49f3-87cc-7076a508e8b5"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:26.063009 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.062989    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c833a520-8403-49f3-87cc-7076a508e8b5-kube-api-access-dfdd2" pod "c833a520-8403-49f3-87cc-7076a508e8b5" (UID: "c833a520-8403-49f3-87cc-7076a508e8b5"). InnerVolumeSpecName "kube-api-access-dfdd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:26.063082 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.062996    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-monitoring-tls-certs" pod "c833a520-8403-49f3-87cc-7076a508e8b5" (UID: "c833a520-8403-49f3-87cc-7076a508e8b5"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:26.161656 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.161631    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c833a520-8403-49f3-87cc-7076a508e8b5-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:26.161656 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.161653    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:26.161772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.161663    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:26.161772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.161672    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfdd2\" (UniqueName: \"kubernetes.io/projected/c833a520-8403-49f3-87cc-7076a508e8b5-kube-api-access-dfdd2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:26.161772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.161683    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c833a520-8403-49f3-87cc-7076a508e8b5-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:26.269639 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.269585    4849 generic.go:356] "Generic (PLEG): container finished" podID="c833a520-8403-49f3-87cc-7076a508e8b5" containerID="b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a" exitCode=0
Oct 30 10:57:26.269639 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.269630    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.269769 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.269655    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c833a520-8403-49f3-87cc-7076a508e8b5","Type":"ContainerDied","Data":"b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a"}
Oct 30 10:57:26.269769 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.269695    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c833a520-8403-49f3-87cc-7076a508e8b5","Type":"ContainerDied","Data":"da6772416f948b8ba64c400314943b6ccd51d2267c9eae26050dfc103fc71500"}
Oct 30 10:57:26.269769 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.269716    4849 scope.go:122] "RemoveContainer" containerID="b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a"
Oct 30 10:57:26.281208 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.281187    4849 scope.go:122] "RemoveContainer" containerID="b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a"
Oct 30 10:57:26.281451 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:26.281434    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a\": container with ID starting with b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a not found: ID does not exist" containerID="b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a"
Oct 30 10:57:26.281499 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.281460    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a"} err="failed to get container status \"b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a\": rpc error: code = NotFound desc = could not find container \"b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a\": container with ID starting with b7bcbe82a1c0859d23f5b983ef24d18345b9ce276f0520de96345c0ca216c80a not found: ID does not exist"
Oct 30 10:57:26.325517 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.325493    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:26.338424 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.338404    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:26.365938 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.365916    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:26.366308 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.366292    4849 cpu_manager.go:414] "containerMap: removing container" podUID="c833a520-8403-49f3-87cc-7076a508e8b5" containerName="otc-container"
Oct 30 10:57:26.366355 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.366308    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c833a520-8403-49f3-87cc-7076a508e8b5" containerName="otc-container"
Oct 30 10:57:26.366399 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.366376    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="c833a520-8403-49f3-87cc-7076a508e8b5" containerName="otc-container"
Oct 30 10:57:26.370739 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.370724    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.388606 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.388587    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:57:26.388705 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.388609    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:57:26.388705 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.388593    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e0aaab7c\""
Oct 30 10:57:26.388705 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.388640    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:57:26.388855 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.388728    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:57:26.390788 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.390769    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:26.464877 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.464854    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.464969 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.464882    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.464969 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.464922    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.465092 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.465074    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.465128 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.465113    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r276\" (UniqueName: \"kubernetes.io/projected/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-kube-api-access-8r276\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.566192 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.566170    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.566298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.566206    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r276\" (UniqueName: \"kubernetes.io/projected/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-kube-api-access-8r276\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.566298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.566230    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.566298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.566248    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.566298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.566278    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.566917 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.566896    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.568757 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.568731    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.568757 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.568747    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.568873 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.568787    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.590740 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.590719    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r276\" (UniqueName: \"kubernetes.io/projected/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-kube-api-access-8r276\") pod \"data-science-collector-collector-0\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.682317 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.682298    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:26.754933 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.754866    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c833a520-8403-49f3-87cc-7076a508e8b5" path="/var/lib/kubelet/pods/c833a520-8403-49f3-87cc-7076a508e8b5/volumes"
Oct 30 10:57:26.983235 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.983212    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:26.985412 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:26.985388    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12b6e4b_9f92_472c_ae75_6f725dfefdc4.slice/crio-0323387e1a01a978aeb3649dc686323531d65b2ed1eb5bc36e8a8c504b15e75d WatchSource:0}: Error finding container 0323387e1a01a978aeb3649dc686323531d65b2ed1eb5bc36e8a8c504b15e75d: Status 404 returned error can't find the container with id 0323387e1a01a978aeb3649dc686323531d65b2ed1eb5bc36e8a8c504b15e75d
Oct 30 10:57:26.987360 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.987326    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:26.987450 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:26.987380    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:27.276958 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:27.276874    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f12b6e4b-9f92-472c-ae75-6f725dfefdc4","Type":"ContainerStarted","Data":"04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67"}
Oct 30 10:57:27.276958 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:27.276944    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f12b6e4b-9f92-472c-ae75-6f725dfefdc4","Type":"ContainerStarted","Data":"0323387e1a01a978aeb3649dc686323531d65b2ed1eb5bc36e8a8c504b15e75d"}
Oct 30 10:57:27.323190 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:27.323141    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.3231277160000001 podStartE2EDuration="1.323127716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:27.320762608 +0000 UTC m=+744.701212048" watchObservedRunningTime="2026-10-30 10:57:27.323127716 +0000 UTC m=+744.703577155"
Oct 30 10:57:31.757286 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:31.757247    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:31.757788 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:31.757476    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="f12b6e4b-9f92-472c-ae75-6f725dfefdc4" containerName="otc-container" containerID="cri-o://04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67" gracePeriod=30
Oct 30 10:57:32.016146 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.016091    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.113488 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.113459    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-data-science-collector-ta-client-cert\") pod \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") "
Oct 30 10:57:32.113624 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.113525    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-tls-certs\") pod \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") "
Oct 30 10:57:32.113624 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.113564    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-otc-internal\" (UniqueName: \"kubernetes.io/configmap/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-otc-internal\") pod \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") "
Oct 30 10:57:32.113624 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.113595    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-kube-api-access-8r276\" (UniqueName: \"kubernetes.io/projected/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-kube-api-access-8r276\") pod \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") "
Oct 30 10:57:32.113725 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.113637    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-monitoring-tls-certs\") pod \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\" (UID: \"f12b6e4b-9f92-472c-ae75-6f725dfefdc4\") "
Oct 30 10:57:32.113973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.113943    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-otc-internal" pod "f12b6e4b-9f92-472c-ae75-6f725dfefdc4" (UID: "f12b6e4b-9f92-472c-ae75-6f725dfefdc4"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:32.115649 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.115625    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-tls-certs" pod "f12b6e4b-9f92-472c-ae75-6f725dfefdc4" (UID: "f12b6e4b-9f92-472c-ae75-6f725dfefdc4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:32.115744 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.115688    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-kube-api-access-8r276" pod "f12b6e4b-9f92-472c-ae75-6f725dfefdc4" (UID: "f12b6e4b-9f92-472c-ae75-6f725dfefdc4"). InnerVolumeSpecName "kube-api-access-8r276". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:32.115812 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.115742    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-monitoring-tls-certs" pod "f12b6e4b-9f92-472c-ae75-6f725dfefdc4" (UID: "f12b6e4b-9f92-472c-ae75-6f725dfefdc4"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:32.115812 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.115767    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-data-science-collector-ta-client-cert" pod "f12b6e4b-9f92-472c-ae75-6f725dfefdc4" (UID: "f12b6e4b-9f92-472c-ae75-6f725dfefdc4"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:32.214314 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.214293    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:32.214314 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.214315    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:32.214449 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.214324    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r276\" (UniqueName: \"kubernetes.io/projected/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-kube-api-access-8r276\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:32.214449 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.214334    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:32.214449 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.214343    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f12b6e4b-9f92-472c-ae75-6f725dfefdc4-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:32.304939 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.304886    4849 generic.go:356] "Generic (PLEG): container finished" podID="f12b6e4b-9f92-472c-ae75-6f725dfefdc4" containerID="04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67" exitCode=0
Oct 30 10:57:32.304939 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.304921    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f12b6e4b-9f92-472c-ae75-6f725dfefdc4","Type":"ContainerDied","Data":"04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67"}
Oct 30 10:57:32.305078 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.304936    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.305078 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.304955    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f12b6e4b-9f92-472c-ae75-6f725dfefdc4","Type":"ContainerDied","Data":"0323387e1a01a978aeb3649dc686323531d65b2ed1eb5bc36e8a8c504b15e75d"}
Oct 30 10:57:32.305078 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.304977    4849 scope.go:122] "RemoveContainer" containerID="04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67"
Oct 30 10:57:32.314212 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.314192    4849 scope.go:122] "RemoveContainer" containerID="04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67"
Oct 30 10:57:32.314481 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:32.314463    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67\": container with ID starting with 04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67 not found: ID does not exist" containerID="04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67"
Oct 30 10:57:32.314531 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.314490    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67"} err="failed to get container status \"04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67\": rpc error: code = NotFound desc = could not find container \"04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67\": container with ID starting with 04ae0e93470c2e4a9b9c600553fbad86955ac2b4056895b5be78873bccfc6d67 not found: ID does not exist"
Oct 30 10:57:32.359073 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.359045    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:32.367203 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.367182    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:32.404627 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.404598    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:32.404980 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.404964    4849 cpu_manager.go:414] "containerMap: removing container" podUID="f12b6e4b-9f92-472c-ae75-6f725dfefdc4" containerName="otc-container"
Oct 30 10:57:32.404980 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.404979    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f12b6e4b-9f92-472c-ae75-6f725dfefdc4" containerName="otc-container"
Oct 30 10:57:32.405152 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.405085    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="f12b6e4b-9f92-472c-ae75-6f725dfefdc4" containerName="otc-container"
Oct 30 10:57:32.409500 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.409480    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.427667 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.427647    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:57:32.427839 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.427817    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:57:32.427931 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.427874    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:57:32.427931 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.427875    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:57:32.427931 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.427913    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:57:32.429724 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.429705    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:32.516577 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.516555    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.516689 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.516608    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.516689 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.516677    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/11f239bd-1764-4223-9496-d800b6d118bc-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.516766 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.516708    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.516766 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.516756    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gn8q\" (UniqueName: \"kubernetes.io/projected/11f239bd-1764-4223-9496-d800b6d118bc-kube-api-access-8gn8q\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.617105 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.617085    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.617201 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.617119    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/11f239bd-1764-4223-9496-d800b6d118bc-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.617201 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.617138    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.617306 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.617245    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gn8q\" (UniqueName: \"kubernetes.io/projected/11f239bd-1764-4223-9496-d800b6d118bc-kube-api-access-8gn8q\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.617376 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.617306    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.617778 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.617753    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/11f239bd-1764-4223-9496-d800b6d118bc-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.619711 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.619691    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.619794 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.619691    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.619794 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.619730    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.647381 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.647360    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gn8q\" (UniqueName: \"kubernetes.io/projected/11f239bd-1764-4223-9496-d800b6d118bc-kube-api-access-8gn8q\") pod \"data-science-collector-collector-0\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.722197 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.722168    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:32.753217 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:32.753189    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f12b6e4b-9f92-472c-ae75-6f725dfefdc4" path="/var/lib/kubelet/pods/f12b6e4b-9f92-472c-ae75-6f725dfefdc4/volumes"
Oct 30 10:57:33.022567 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:33.022448    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:33.025216 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:33.025190    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f239bd_1764_4223_9496_d800b6d118bc.slice/crio-409c742590ccf4ac4025f9d0cff9198536aed3da69b4ce04de277187f4fae6dc WatchSource:0}: Error finding container 409c742590ccf4ac4025f9d0cff9198536aed3da69b4ce04de277187f4fae6dc: Status 404 returned error can't find the container with id 409c742590ccf4ac4025f9d0cff9198536aed3da69b4ce04de277187f4fae6dc
Oct 30 10:57:33.027344 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:33.027313    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:33.027438 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:33.027368    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:33.313692 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:33.313659    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"11f239bd-1764-4223-9496-d800b6d118bc","Type":"ContainerStarted","Data":"d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6"}
Oct 30 10:57:33.313849 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:33.313710    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"11f239bd-1764-4223-9496-d800b6d118bc","Type":"ContainerStarted","Data":"409c742590ccf4ac4025f9d0cff9198536aed3da69b4ce04de277187f4fae6dc"}
Oct 30 10:57:33.358938 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:33.358891    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.358877241 podStartE2EDuration="1.358877241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:33.357588164 +0000 UTC m=+750.738037604" watchObservedRunningTime="2026-10-30 10:57:33.358877241 +0000 UTC m=+750.739326679"
Oct 30 10:57:39.430107 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:39.430067    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:57:40.229541 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.229507    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:40.233701 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.233678    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.250076 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.250051    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-ca-bundle\""
Oct 30 10:57:40.250076 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.250051    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:57:40.250246 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.250230    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-serving-cabundle\""
Oct 30 10:57:40.250287 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.250229    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-config\""
Oct 30 10:57:40.265664 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.265640    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:40.275744 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.275718    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.275834 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.275775    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.275883 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.275833    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.275916 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.275895    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbwcq\" (UniqueName: \"kubernetes.io/projected/65855a4e-66ae-4138-92ee-f63e6a9407d0-kube-api-access-rbwcq\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.275973 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.275947    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.276034 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.275979    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.276079 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.276058    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.276158 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.276135    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.298889 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.298864    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway-serving-cert\""
Oct 30 10:57:40.298983 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.298919    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-dockercfg-26zr8\""
Oct 30 10:57:40.298983 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.298917    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 10:57:40.299103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.298980    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-mtls\""
Oct 30 10:57:40.377117 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377099    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377217 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377133    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377217 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377159    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377286 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:40.377269    4849 secret.go:189] Couldn't get secret redhat-ods-monitoring/tempo-data-science-tempomonolithic-gateway-serving-cert: secret "tempo-data-science-tempomonolithic-gateway-serving-cert" not found
Oct 30 10:57:40.377343 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:40.377332    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert podName:65855a4e-66ae-4138-92ee-f63e6a9407d0 nodeName:}" failed. No retries permitted until 2026-10-30 10:57:40.877304904 +0000 UTC m=+758.257754322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tempo-data-science-tempomonolithic-gateway-serving-cert" (UniqueName: "kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert") pod "tempo-data-science-tempomonolithic-0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0") : secret "tempo-data-science-tempomonolithic-gateway-serving-cert" not found
Oct 30 10:57:40.377394 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377352    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbwcq\" (UniqueName: \"kubernetes.io/projected/65855a4e-66ae-4138-92ee-f63e6a9407d0-kube-api-access-rbwcq\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377448 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377423    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377496 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377473    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377552 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377507    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377611 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377550    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377924 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377900    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.377989 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.377927    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.378166 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.378145    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.378261 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.378234    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.380164 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.380142    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.380241 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.380202    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.401715 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.401691    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbwcq\" (UniqueName: \"kubernetes.io/projected/65855a4e-66ae-4138-92ee-f63e6a9407d0-kube-api-access-rbwcq\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.808244 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.808210    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:40.808674 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.808482    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="11f239bd-1764-4223-9496-d800b6d118bc" containerName="otc-container" containerID="cri-o://d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6" gracePeriod=30
Oct 30 10:57:40.881548 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.881519    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:40.883806 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:40.883786    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:41.047730 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.047704    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.083708 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.083642    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-monitoring-tls-certs\") pod \"11f239bd-1764-4223-9496-d800b6d118bc\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") "
Oct 30 10:57:41.083708 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.083686    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11f239bd-1764-4223-9496-d800b6d118bc-kube-api-access-8gn8q\" (UniqueName: \"kubernetes.io/projected/11f239bd-1764-4223-9496-d800b6d118bc-kube-api-access-8gn8q\") pod \"11f239bd-1764-4223-9496-d800b6d118bc\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") "
Oct 30 10:57:41.083888 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.083733    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11f239bd-1764-4223-9496-d800b6d118bc-otc-internal\" (UniqueName: \"kubernetes.io/configmap/11f239bd-1764-4223-9496-d800b6d118bc-otc-internal\") pod \"11f239bd-1764-4223-9496-d800b6d118bc\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") "
Oct 30 10:57:41.083888 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.083843    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-data-science-collector-ta-client-cert\") pod \"11f239bd-1764-4223-9496-d800b6d118bc\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") "
Oct 30 10:57:41.084001 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.083885    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-tls-certs\") pod \"11f239bd-1764-4223-9496-d800b6d118bc\" (UID: \"11f239bd-1764-4223-9496-d800b6d118bc\") "
Oct 30 10:57:41.084208 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.084179    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f239bd-1764-4223-9496-d800b6d118bc-otc-internal" pod "11f239bd-1764-4223-9496-d800b6d118bc" (UID: "11f239bd-1764-4223-9496-d800b6d118bc"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:41.084322 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.084299    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/11f239bd-1764-4223-9496-d800b6d118bc-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:41.085778 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.085753    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f239bd-1764-4223-9496-d800b6d118bc-kube-api-access-8gn8q" pod "11f239bd-1764-4223-9496-d800b6d118bc" (UID: "11f239bd-1764-4223-9496-d800b6d118bc"). InnerVolumeSpecName "kube-api-access-8gn8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:41.085984 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.085950    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-data-science-collector-ta-client-cert" pod "11f239bd-1764-4223-9496-d800b6d118bc" (UID: "11f239bd-1764-4223-9496-d800b6d118bc"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:41.086056 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.085976    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-monitoring-tls-certs" pod "11f239bd-1764-4223-9496-d800b6d118bc" (UID: "11f239bd-1764-4223-9496-d800b6d118bc"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:41.086056 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.086009    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-tls-certs" pod "11f239bd-1764-4223-9496-d800b6d118bc" (UID: "11f239bd-1764-4223-9496-d800b6d118bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:41.185113 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.185083    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:41.185113 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.185113    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:41.185277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.185127    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/11f239bd-1764-4223-9496-d800b6d118bc-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:41.185277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.185139    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gn8q\" (UniqueName: \"kubernetes.io/projected/11f239bd-1764-4223-9496-d800b6d118bc-kube-api-access-8gn8q\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:41.358550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.358480    4849 generic.go:356] "Generic (PLEG): container finished" podID="11f239bd-1764-4223-9496-d800b6d118bc" containerID="d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6" exitCode=0
Oct 30 10:57:41.358550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.358528    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.358667 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.358567    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"11f239bd-1764-4223-9496-d800b6d118bc","Type":"ContainerDied","Data":"d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6"}
Oct 30 10:57:41.358667 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.358618    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"11f239bd-1764-4223-9496-d800b6d118bc","Type":"ContainerDied","Data":"409c742590ccf4ac4025f9d0cff9198536aed3da69b4ce04de277187f4fae6dc"}
Oct 30 10:57:41.358667 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.358640    4849 scope.go:122] "RemoveContainer" containerID="d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6"
Oct 30 10:57:41.371500 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.371480    4849 scope.go:122] "RemoveContainer" containerID="d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6"
Oct 30 10:57:41.371747 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:41.371730    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6\": container with ID starting with d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6 not found: ID does not exist" containerID="d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6"
Oct 30 10:57:41.371792 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.371754    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6"} err="failed to get container status \"d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6\": rpc error: code = NotFound desc = could not find container \"d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6\": container with ID starting with d5d15cc689959eee6d0d8eaeb2f461f0b3de00c411a0e2c195eaea5eb4632bd6 not found: ID does not exist"
Oct 30 10:57:41.415490 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.414490    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:41.427231 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.427208    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:41.455410 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.455390    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:41.455806 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.455793    4849 cpu_manager.go:414] "containerMap: removing container" podUID="11f239bd-1764-4223-9496-d800b6d118bc" containerName="otc-container"
Oct 30 10:57:41.455857 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.455809    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11f239bd-1764-4223-9496-d800b6d118bc" containerName="otc-container"
Oct 30 10:57:41.455888 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.455871    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="11f239bd-1764-4223-9496-d800b6d118bc" containerName="otc-container"
Oct 30 10:57:41.460422 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.460408    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.478512 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.478492    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:57:41.478618 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.478546    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:57:41.478756 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.478734    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-4873cf1a\""
Oct 30 10:57:41.478811 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.478789    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:57:41.478873 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.478859    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:57:41.481149 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.481124    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:41.487497 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.487472    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.487595 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.487525    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.487658 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.487591    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6pkc\" (UniqueName: \"kubernetes.io/projected/78e986dd-5e4f-4341-9655-039345346ebe-kube-api-access-f6pkc\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.487658 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.487641    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78e986dd-5e4f-4341-9655-039345346ebe-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.487762 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.487674    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.588396 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.588377    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.588499 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.588412    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6pkc\" (UniqueName: \"kubernetes.io/projected/78e986dd-5e4f-4341-9655-039345346ebe-kube-api-access-f6pkc\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.588499 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.588433    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78e986dd-5e4f-4341-9655-039345346ebe-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.588499 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.588449    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.588499 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.588483    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.589180 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.589160    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78e986dd-5e4f-4341-9655-039345346ebe-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.590993 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.590970    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.590993 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.590987    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.591136 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.590964    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.612544 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.612500    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6pkc\" (UniqueName: \"kubernetes.io/projected/78e986dd-5e4f-4341-9655-039345346ebe-kube-api-access-f6pkc\") pod \"data-science-collector-collector-0\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:41.772363 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:41.772344    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:42.070581 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:42.070516    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:42.072976 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:42.072951    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e986dd_5e4f_4341_9655_039345346ebe.slice/crio-f7fcf32d0bdc2acc6876d0c3975ac4eb8ba461c974da926f0909a02fbe4f58bb WatchSource:0}: Error finding container f7fcf32d0bdc2acc6876d0c3975ac4eb8ba461c974da926f0909a02fbe4f58bb: Status 404 returned error can't find the container with id f7fcf32d0bdc2acc6876d0c3975ac4eb8ba461c974da926f0909a02fbe4f58bb
Oct 30 10:57:42.075016 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:42.074986    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:42.075119 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:42.075056    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:42.366375 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:42.366293    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"78e986dd-5e4f-4341-9655-039345346ebe","Type":"ContainerStarted","Data":"a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62"}
Oct 30 10:57:42.366375 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:42.366357    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"78e986dd-5e4f-4341-9655-039345346ebe","Type":"ContainerStarted","Data":"f7fcf32d0bdc2acc6876d0c3975ac4eb8ba461c974da926f0909a02fbe4f58bb"}
Oct 30 10:57:42.412013 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:42.411968    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.4119554380000001 podStartE2EDuration="1.411955438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:42.41012892 +0000 UTC m=+759.790578385" watchObservedRunningTime="2026-10-30 10:57:42.411955438 +0000 UTC m=+759.792404877"
Oct 30 10:57:42.754467 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:42.754394    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11f239bd-1764-4223-9496-d800b6d118bc" path="/var/lib/kubelet/pods/11f239bd-1764-4223-9496-d800b6d118bc/volumes"
Oct 30 10:57:48.179206 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.179167    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:48.194858 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:48.194826    4849 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[tempo-storage], unattached volumes=[tempo-storage], failed to process volumes=[]: context canceled" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="65855a4e-66ae-4138-92ee-f63e6a9407d0"
Oct 30 10:57:48.398538 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.398504    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:48.405821 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.405794    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:48.544238 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544171    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544238 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544207    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-rbac\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544406 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544252    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544406 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544277    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-ca-bundle\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544406 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544312    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65855a4e-66ae-4138-92ee-f63e6a9407d0-kube-api-access-rbwcq\" (UniqueName: \"kubernetes.io/projected/65855a4e-66ae-4138-92ee-f63e6a9407d0-kube-api-access-rbwcq\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544569 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544439    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-tenants\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-tenants\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544569 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544510    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-mtls\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544680 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544587    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-conf\") pod \"65855a4e-66ae-4138-92ee-f63e6a9407d0\" (UID: \"65855a4e-66ae-4138-92ee-f63e6a9407d0\") "
Oct 30 10:57:48.544680 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544645    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-rbac" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "gateway-rbac". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:48.544680 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544663    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-serving-cabundle" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-serving-cabundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:48.544822 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544684    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-ca-bundle" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:48.544918 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544894    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-serving-cabundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.544996 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544920    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.544996 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.544950    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-rbac\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.545145 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.545001    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-conf" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:48.546560 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.546528    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-gateway-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:48.546685 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.546628    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-mtls" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:48.546685 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.546675    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-tenants" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "gateway-tenants". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:48.546785 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.546704    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65855a4e-66ae-4138-92ee-f63e6a9407d0-kube-api-access-rbwcq" pod "65855a4e-66ae-4138-92ee-f63e6a9407d0" (UID: "65855a4e-66ae-4138-92ee-f63e6a9407d0"). InnerVolumeSpecName "kube-api-access-rbwcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:48.645487 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.645466    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbwcq\" (UniqueName: \"kubernetes.io/projected/65855a4e-66ae-4138-92ee-f63e6a9407d0-kube-api-access-rbwcq\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.645487 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.645486    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-gateway-tenants\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.645675 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.645496    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.645675 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.645505    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.645675 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.645514    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/65855a4e-66ae-4138-92ee-f63e6a9407d0-tempo-data-science-tempomonolithic-gateway-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:48.863858 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.863823    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:48.864445 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:48.864376    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="78e986dd-5e4f-4341-9655-039345346ebe" containerName="otc-container" containerID="cri-o://a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62" gracePeriod=30
Oct 30 10:57:49.403822 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:49.403796    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 10:57:49.490980 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:49.490952    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:49.497296 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:49.497241    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 10:57:50.116820 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.116797    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.257858 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.257782    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78e986dd-5e4f-4341-9655-039345346ebe-kube-api-access-f6pkc\" (UniqueName: \"kubernetes.io/projected/78e986dd-5e4f-4341-9655-039345346ebe-kube-api-access-f6pkc\") pod \"78e986dd-5e4f-4341-9655-039345346ebe\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") "
Oct 30 10:57:50.257858 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.257822    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-monitoring-tls-certs\") pod \"78e986dd-5e4f-4341-9655-039345346ebe\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") "
Oct 30 10:57:50.258099 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.257865    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78e986dd-5e4f-4341-9655-039345346ebe-otc-internal\" (UniqueName: \"kubernetes.io/configmap/78e986dd-5e4f-4341-9655-039345346ebe-otc-internal\") pod \"78e986dd-5e4f-4341-9655-039345346ebe\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") "
Oct 30 10:57:50.258099 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.257895    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-tls-certs\") pod \"78e986dd-5e4f-4341-9655-039345346ebe\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") "
Oct 30 10:57:50.258099 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.257948    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-data-science-collector-ta-client-cert\") pod \"78e986dd-5e4f-4341-9655-039345346ebe\" (UID: \"78e986dd-5e4f-4341-9655-039345346ebe\") "
Oct 30 10:57:50.258322 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.258297    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e986dd-5e4f-4341-9655-039345346ebe-otc-internal" pod "78e986dd-5e4f-4341-9655-039345346ebe" (UID: "78e986dd-5e4f-4341-9655-039345346ebe"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:50.259929 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.259898    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-monitoring-tls-certs" pod "78e986dd-5e4f-4341-9655-039345346ebe" (UID: "78e986dd-5e4f-4341-9655-039345346ebe"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:50.260062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.259932    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-data-science-collector-ta-client-cert" pod "78e986dd-5e4f-4341-9655-039345346ebe" (UID: "78e986dd-5e4f-4341-9655-039345346ebe"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:50.260062 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.260011    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-tls-certs" pod "78e986dd-5e4f-4341-9655-039345346ebe" (UID: "78e986dd-5e4f-4341-9655-039345346ebe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:50.260143 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.260076    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e986dd-5e4f-4341-9655-039345346ebe-kube-api-access-f6pkc" pod "78e986dd-5e4f-4341-9655-039345346ebe" (UID: "78e986dd-5e4f-4341-9655-039345346ebe"). InnerVolumeSpecName "kube-api-access-f6pkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:50.359103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.359061    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6pkc\" (UniqueName: \"kubernetes.io/projected/78e986dd-5e4f-4341-9655-039345346ebe-kube-api-access-f6pkc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:50.359103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.359084    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:50.359103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.359093    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78e986dd-5e4f-4341-9655-039345346ebe-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:50.359103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.359102    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:50.359302 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.359112    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/78e986dd-5e4f-4341-9655-039345346ebe-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:50.410416 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.410383    4849 generic.go:356] "Generic (PLEG): container finished" podID="78e986dd-5e4f-4341-9655-039345346ebe" containerID="a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62" exitCode=0
Oct 30 10:57:50.410806 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.410433    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.410806 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.410473    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"78e986dd-5e4f-4341-9655-039345346ebe","Type":"ContainerDied","Data":"a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62"}
Oct 30 10:57:50.410806 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.410542    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"78e986dd-5e4f-4341-9655-039345346ebe","Type":"ContainerDied","Data":"f7fcf32d0bdc2acc6876d0c3975ac4eb8ba461c974da926f0909a02fbe4f58bb"}
Oct 30 10:57:50.410806 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.410574    4849 scope.go:122] "RemoveContainer" containerID="a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62"
Oct 30 10:57:50.419575 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.419555    4849 scope.go:122] "RemoveContainer" containerID="a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62"
Oct 30 10:57:50.419840 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:50.419822    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62\": container with ID starting with a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62 not found: ID does not exist" containerID="a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62"
Oct 30 10:57:50.419901 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.419852    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62"} err="failed to get container status \"a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62\": rpc error: code = NotFound desc = could not find container \"a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62\": container with ID starting with a8a690a3cdf7d098a8e6c6d195ce68af0b5a129824dfb9c80ca3ac2411d4df62 not found: ID does not exist"
Oct 30 10:57:50.465613 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.465591    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:50.479068 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.479044    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:50.505462 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.505441    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:50.505808 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.505797    4849 cpu_manager.go:414] "containerMap: removing container" podUID="78e986dd-5e4f-4341-9655-039345346ebe" containerName="otc-container"
Oct 30 10:57:50.505857 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.505835    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78e986dd-5e4f-4341-9655-039345346ebe" containerName="otc-container"
Oct 30 10:57:50.505933 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.505924    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="78e986dd-5e4f-4341-9655-039345346ebe" containerName="otc-container"
Oct 30 10:57:50.510175 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.510129    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.529382 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.529357    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:57:50.530071 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.530048    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:57:50.530174 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.530076    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:57:50.530174 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.530100    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:57:50.530174 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.530105    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:57:50.532236 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.532217    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:50.661380 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.661355    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.661477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.661401    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcd4v\" (UniqueName: \"kubernetes.io/projected/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-kube-api-access-jcd4v\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.661477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.661469    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.661549 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.661496    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.661549 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.661515    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.753388 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.753368    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65855a4e-66ae-4138-92ee-f63e6a9407d0" path="/var/lib/kubelet/pods/65855a4e-66ae-4138-92ee-f63e6a9407d0/volumes"
Oct 30 10:57:50.753650 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.753638    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78e986dd-5e4f-4341-9655-039345346ebe" path="/var/lib/kubelet/pods/78e986dd-5e4f-4341-9655-039345346ebe/volumes"
Oct 30 10:57:50.762689 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.762641    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.762689 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.762667    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.762856 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.762709    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.762856 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.762825    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcd4v\" (UniqueName: \"kubernetes.io/projected/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-kube-api-access-jcd4v\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.762964 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.762861    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.763316 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.763294    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.764975 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.764955    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.765083 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.765068    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.765083 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.765074    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.786864 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.786845    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcd4v\" (UniqueName: \"kubernetes.io/projected/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-kube-api-access-jcd4v\") pod \"data-science-collector-collector-0\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:50.822666 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:50.822648    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:51.121871 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:51.121830    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:51.124177 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:51.124145    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ecd4f41_2ddf_48fd_8d1c_a1c5f7c36f7a.slice/crio-bc85a3a59c94058bf51fbd5e753d5335e7006380d213c00f69f638881be26cfb WatchSource:0}: Error finding container bc85a3a59c94058bf51fbd5e753d5335e7006380d213c00f69f638881be26cfb: Status 404 returned error can't find the container with id bc85a3a59c94058bf51fbd5e753d5335e7006380d213c00f69f638881be26cfb
Oct 30 10:57:51.126400 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:51.126372    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:51.126451 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:51.126427    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:51.417997 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:51.417918    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a","Type":"ContainerStarted","Data":"0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a"}
Oct 30 10:57:51.417997 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:51.417968    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a","Type":"ContainerStarted","Data":"bc85a3a59c94058bf51fbd5e753d5335e7006380d213c00f69f638881be26cfb"}
Oct 30 10:57:51.463089 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:51.463037    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.463006851 podStartE2EDuration="1.463006851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:51.462103314 +0000 UTC m=+768.842552752" watchObservedRunningTime="2026-10-30 10:57:51.463006851 +0000 UTC m=+768.843456289"
Oct 30 10:57:57.907469 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:57.907440    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:57.907902 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:57.907605    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" containerName="otc-container" containerID="cri-o://0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a" gracePeriod=30
Oct 30 10:57:58.162285 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.162226    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.318805 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.318774    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-kube-api-access-jcd4v\" (UniqueName: \"kubernetes.io/projected/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-kube-api-access-jcd4v\") pod \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") "
Oct 30 10:57:58.318938 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.318816    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-otc-internal\" (UniqueName: \"kubernetes.io/configmap/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-otc-internal\") pod \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") "
Oct 30 10:57:58.318938 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.318857    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-data-science-collector-ta-client-cert\") pod \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") "
Oct 30 10:57:58.318938 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.318882    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-monitoring-tls-certs\") pod \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") "
Oct 30 10:57:58.318938 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.318921    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-tls-certs\") pod \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\" (UID: \"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a\") "
Oct 30 10:57:58.319302 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.319274    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-otc-internal" pod "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" (UID: "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:57:58.321001 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.320974    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-monitoring-tls-certs" pod "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" (UID: "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:58.321001 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.320987    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-kube-api-access-jcd4v" pod "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" (UID: "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a"). InnerVolumeSpecName "kube-api-access-jcd4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:57:58.321167 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.321123    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-tls-certs" pod "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" (UID: "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:58.321210 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.321174    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-data-science-collector-ta-client-cert" pod "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" (UID: "9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:57:58.420328 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.420276    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcd4v\" (UniqueName: \"kubernetes.io/projected/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-kube-api-access-jcd4v\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:58.420328 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.420296    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:58.420328 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.420307    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:58.420328 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.420315    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:58.420328 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.420324    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:57:58.458708 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.458682    4849 generic.go:356] "Generic (PLEG): container finished" podID="9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" containerID="0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a" exitCode=0
Oct 30 10:57:58.458816 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.458727    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.458816 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.458763    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a","Type":"ContainerDied","Data":"0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a"}
Oct 30 10:57:58.458816 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.458813    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a","Type":"ContainerDied","Data":"bc85a3a59c94058bf51fbd5e753d5335e7006380d213c00f69f638881be26cfb"}
Oct 30 10:57:58.458936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.458838    4849 scope.go:122] "RemoveContainer" containerID="0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a"
Oct 30 10:57:58.467778 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.467763    4849 scope.go:122] "RemoveContainer" containerID="0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a"
Oct 30 10:57:58.468045 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:57:58.468002    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a\": container with ID starting with 0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a not found: ID does not exist" containerID="0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a"
Oct 30 10:57:58.468103 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.468055    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a"} err="failed to get container status \"0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a\": rpc error: code = NotFound desc = could not find container \"0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a\": container with ID starting with 0a746abd3670edcf09f790aa5582b4fb40d779a9e9963c14eaf2cf2676eb0b5a not found: ID does not exist"
Oct 30 10:57:58.513788 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.513766    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:58.527910 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.527884    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:58.557277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.554454    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:58.557277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.555356    4849 cpu_manager.go:414] "containerMap: removing container" podUID="9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" containerName="otc-container"
Oct 30 10:57:58.557277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.555374    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" containerName="otc-container"
Oct 30 10:57:58.557277 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.555564    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" containerName="otc-container"
Oct 30 10:57:58.561239 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.561214    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.579309 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.579275    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:58.584797 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.584773    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:57:58.585228 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.585184    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:57:58.585362 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.585201    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:57:58.585362 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.585235    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:57:58.585464 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.585368    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:57:58.723343 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.723293    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.723343 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.723324    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.723343 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.723339    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjvk5\" (UniqueName: \"kubernetes.io/projected/35859f88-4ba8-40b5-880b-79cdf7265fe8-kube-api-access-jjvk5\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.723561 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.723426    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.723561 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.723478    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/35859f88-4ba8-40b5-880b-79cdf7265fe8-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.753085 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.753063    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a" path="/var/lib/kubelet/pods/9ecd4f41-2ddf-48fd-8d1c-a1c5f7c36f7a/volumes"
Oct 30 10:57:58.823989 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.823968    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.824126 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.823995    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/35859f88-4ba8-40b5-880b-79cdf7265fe8-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.824126 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.824045    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.824126 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.824107    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.824298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.824145    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjvk5\" (UniqueName: \"kubernetes.io/projected/35859f88-4ba8-40b5-880b-79cdf7265fe8-kube-api-access-jjvk5\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.824747 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.824717    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/35859f88-4ba8-40b5-880b-79cdf7265fe8-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.826528 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.826510    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.826607 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.826563    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.826607 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.826578    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.848551 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.848528    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjvk5\" (UniqueName: \"kubernetes.io/projected/35859f88-4ba8-40b5-880b-79cdf7265fe8-kube-api-access-jjvk5\") pod \"data-science-collector-collector-0\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:58.874849 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:58.874829    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:57:59.173552 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:59.173530    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:57:59.175778 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:57:59.175752    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35859f88_4ba8_40b5_880b_79cdf7265fe8.slice/crio-3b40ffb696177ff15abb8ed240715dd78fedf5ecf4a8e572e82ab9ba3d536cb3 WatchSource:0}: Error finding container 3b40ffb696177ff15abb8ed240715dd78fedf5ecf4a8e572e82ab9ba3d536cb3: Status 404 returned error can't find the container with id 3b40ffb696177ff15abb8ed240715dd78fedf5ecf4a8e572e82ab9ba3d536cb3
Oct 30 10:57:59.177701 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:59.177673    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:59.177788 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:59.177727    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:57:59.467076 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:59.466978    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"35859f88-4ba8-40b5-880b-79cdf7265fe8","Type":"ContainerStarted","Data":"e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d"}
Oct 30 10:57:59.467076 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:59.467043    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"35859f88-4ba8-40b5-880b-79cdf7265fe8","Type":"ContainerStarted","Data":"3b40ffb696177ff15abb8ed240715dd78fedf5ecf4a8e572e82ab9ba3d536cb3"}
Oct 30 10:57:59.511979 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:57:59.511934    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.511921622 podStartE2EDuration="1.511921622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:57:59.511165848 +0000 UTC m=+776.891615289" watchObservedRunningTime="2026-10-30 10:57:59.511921622 +0000 UTC m=+776.892371061"
Oct 30 10:58:06.945641 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:06.945605    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:06.946165 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:06.945871    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="35859f88-4ba8-40b5-880b-79cdf7265fe8" containerName="otc-container" containerID="cri-o://e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d" gracePeriod=30
Oct 30 10:58:07.197189 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.197137    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.294472 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.294442    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-monitoring-tls-certs\") pod \"35859f88-4ba8-40b5-880b-79cdf7265fe8\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") "
Oct 30 10:58:07.294639 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.294505    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35859f88-4ba8-40b5-880b-79cdf7265fe8-otc-internal\" (UniqueName: \"kubernetes.io/configmap/35859f88-4ba8-40b5-880b-79cdf7265fe8-otc-internal\") pod \"35859f88-4ba8-40b5-880b-79cdf7265fe8\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") "
Oct 30 10:58:07.294639 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.294529    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-tls-certs\") pod \"35859f88-4ba8-40b5-880b-79cdf7265fe8\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") "
Oct 30 10:58:07.294773 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.294735    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35859f88-4ba8-40b5-880b-79cdf7265fe8-kube-api-access-jjvk5\" (UniqueName: \"kubernetes.io/projected/35859f88-4ba8-40b5-880b-79cdf7265fe8-kube-api-access-jjvk5\") pod \"35859f88-4ba8-40b5-880b-79cdf7265fe8\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") "
Oct 30 10:58:07.294828 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.294813    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-data-science-collector-ta-client-cert\") pod \"35859f88-4ba8-40b5-880b-79cdf7265fe8\" (UID: \"35859f88-4ba8-40b5-880b-79cdf7265fe8\") "
Oct 30 10:58:07.295165 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.294934    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35859f88-4ba8-40b5-880b-79cdf7265fe8-otc-internal" pod "35859f88-4ba8-40b5-880b-79cdf7265fe8" (UID: "35859f88-4ba8-40b5-880b-79cdf7265fe8"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:58:07.295279 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.295174    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/35859f88-4ba8-40b5-880b-79cdf7265fe8-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:07.296834 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.296808    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-monitoring-tls-certs" pod "35859f88-4ba8-40b5-880b-79cdf7265fe8" (UID: "35859f88-4ba8-40b5-880b-79cdf7265fe8"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:07.297207 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.297175    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-tls-certs" pod "35859f88-4ba8-40b5-880b-79cdf7265fe8" (UID: "35859f88-4ba8-40b5-880b-79cdf7265fe8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:07.297207 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.297182    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35859f88-4ba8-40b5-880b-79cdf7265fe8-kube-api-access-jjvk5" pod "35859f88-4ba8-40b5-880b-79cdf7265fe8" (UID: "35859f88-4ba8-40b5-880b-79cdf7265fe8"). InnerVolumeSpecName "kube-api-access-jjvk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:58:07.297380 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.297360    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-data-science-collector-ta-client-cert" pod "35859f88-4ba8-40b5-880b-79cdf7265fe8" (UID: "35859f88-4ba8-40b5-880b-79cdf7265fe8"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:07.396215 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.396195    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:07.396215 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.396214    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjvk5\" (UniqueName: \"kubernetes.io/projected/35859f88-4ba8-40b5-880b-79cdf7265fe8-kube-api-access-jjvk5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:07.396355 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.396225    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:07.396355 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.396234    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/35859f88-4ba8-40b5-880b-79cdf7265fe8-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:07.510269 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.510216    4849 generic.go:356] "Generic (PLEG): container finished" podID="35859f88-4ba8-40b5-880b-79cdf7265fe8" containerID="e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d" exitCode=0
Oct 30 10:58:07.510353 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.510267    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.510353 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.510286    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"35859f88-4ba8-40b5-880b-79cdf7265fe8","Type":"ContainerDied","Data":"e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d"}
Oct 30 10:58:07.510353 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.510328    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"35859f88-4ba8-40b5-880b-79cdf7265fe8","Type":"ContainerDied","Data":"3b40ffb696177ff15abb8ed240715dd78fedf5ecf4a8e572e82ab9ba3d536cb3"}
Oct 30 10:58:07.510353 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.510347    4849 scope.go:122] "RemoveContainer" containerID="e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d"
Oct 30 10:58:07.519259 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.519241    4849 scope.go:122] "RemoveContainer" containerID="e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d"
Oct 30 10:58:07.519534 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:58:07.519512    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d\": container with ID starting with e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d not found: ID does not exist" containerID="e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d"
Oct 30 10:58:07.519574 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.519542    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d"} err="failed to get container status \"e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d\": rpc error: code = NotFound desc = could not find container \"e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d\": container with ID starting with e60aadee3cff36f6685776edf8839d5bfa0852690df23774474a28b3f9fbb42d not found: ID does not exist"
Oct 30 10:58:07.566139 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.566116    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:07.579454 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.579430    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:07.613809 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.613788    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:07.614217 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.614197    4849 cpu_manager.go:414] "containerMap: removing container" podUID="35859f88-4ba8-40b5-880b-79cdf7265fe8" containerName="otc-container"
Oct 30 10:58:07.614217 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.614213    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35859f88-4ba8-40b5-880b-79cdf7265fe8" containerName="otc-container"
Oct 30 10:58:07.614338 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.614314    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="35859f88-4ba8-40b5-880b-79cdf7265fe8" containerName="otc-container"
Oct 30 10:58:07.618725 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.618708    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.636941 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.636919    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:58:07.637476 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.637454    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:58:07.637582 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.637560    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:58:07.637648 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.637609    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:58:07.637700 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.637647    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:58:07.639495 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.639474    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:07.699083 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.699058    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.699185 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.699102    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.699269 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.699250    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/3afca0a5-ee7e-4c95-80b6-441fea71ade3-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.699343 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.699296    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.699343 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.699322    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh8g7\" (UniqueName: \"kubernetes.io/projected/3afca0a5-ee7e-4c95-80b6-441fea71ade3-kube-api-access-hh8g7\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.800331 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.800279    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/3afca0a5-ee7e-4c95-80b6-441fea71ade3-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.800331 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.800310    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.800331 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.800327    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh8g7\" (UniqueName: \"kubernetes.io/projected/3afca0a5-ee7e-4c95-80b6-441fea71ade3-kube-api-access-hh8g7\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.800550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.800361    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.800550 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.800403    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.800979 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.800954    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/3afca0a5-ee7e-4c95-80b6-441fea71ade3-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.802874 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.802847    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.802946 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.802891    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.802996 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.802973    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.824815 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.824796    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh8g7\" (UniqueName: \"kubernetes.io/projected/3afca0a5-ee7e-4c95-80b6-441fea71ade3-kube-api-access-hh8g7\") pod \"data-science-collector-collector-0\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:07.930660 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:07.930635    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:08.228003 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:08.227976    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:08.230506 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:58:08.230480    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3afca0a5_ee7e_4c95_80b6_441fea71ade3.slice/crio-2fee28dd528f0e44e882b92d6d2642febdf3174214ae95d7c0e29a503999f715 WatchSource:0}: Error finding container 2fee28dd528f0e44e882b92d6d2642febdf3174214ae95d7c0e29a503999f715: Status 404 returned error can't find the container with id 2fee28dd528f0e44e882b92d6d2642febdf3174214ae95d7c0e29a503999f715
Oct 30 10:58:08.232502 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:08.232473    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:58:08.232575 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:08.232533    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:58:08.517345 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:08.517230    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"3afca0a5-ee7e-4c95-80b6-441fea71ade3","Type":"ContainerStarted","Data":"c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105"}
Oct 30 10:58:08.517345 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:08.517297    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"3afca0a5-ee7e-4c95-80b6-441fea71ade3","Type":"ContainerStarted","Data":"2fee28dd528f0e44e882b92d6d2642febdf3174214ae95d7c0e29a503999f715"}
Oct 30 10:58:08.564473 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:08.564420    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.564404312 podStartE2EDuration="1.564404312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:58:08.563124913 +0000 UTC m=+785.943574349" watchObservedRunningTime="2026-10-30 10:58:08.564404312 +0000 UTC m=+785.944853754"
Oct 30 10:58:08.752870 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:08.752837    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35859f88-4ba8-40b5-880b-79cdf7265fe8" path="/var/lib/kubelet/pods/35859f88-4ba8-40b5-880b-79cdf7265fe8/volumes"
Oct 30 10:58:09.431875 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:58:09.431843    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:58:22.029668 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:22.029625    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:22.030298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:22.029870    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="3afca0a5-ee7e-4c95-80b6-441fea71ade3" containerName="otc-container" containerID="cri-o://c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105" gracePeriod=30
Oct 30 10:58:23.268054 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.268015    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.320428 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.320400    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-tls-certs\") pod \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") "
Oct 30 10:58:23.320546 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.320471    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3afca0a5-ee7e-4c95-80b6-441fea71ade3-kube-api-access-hh8g7\" (UniqueName: \"kubernetes.io/projected/3afca0a5-ee7e-4c95-80b6-441fea71ade3-kube-api-access-hh8g7\") pod \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") "
Oct 30 10:58:23.320669 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.320545    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-monitoring-tls-certs\") pod \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") "
Oct 30 10:58:23.320669 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.320600    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3afca0a5-ee7e-4c95-80b6-441fea71ade3-otc-internal\" (UniqueName: \"kubernetes.io/configmap/3afca0a5-ee7e-4c95-80b6-441fea71ade3-otc-internal\") pod \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") "
Oct 30 10:58:23.320669 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.320635    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-data-science-collector-ta-client-cert\") pod \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\" (UID: \"3afca0a5-ee7e-4c95-80b6-441fea71ade3\") "
Oct 30 10:58:23.321117 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.321081    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afca0a5-ee7e-4c95-80b6-441fea71ade3-otc-internal" pod "3afca0a5-ee7e-4c95-80b6-441fea71ade3" (UID: "3afca0a5-ee7e-4c95-80b6-441fea71ade3"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:58:23.321258 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.321223    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/3afca0a5-ee7e-4c95-80b6-441fea71ade3-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:23.323163 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.323132    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-monitoring-tls-certs" pod "3afca0a5-ee7e-4c95-80b6-441fea71ade3" (UID: "3afca0a5-ee7e-4c95-80b6-441fea71ade3"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:23.323311 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.323270    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3afca0a5-ee7e-4c95-80b6-441fea71ade3-kube-api-access-hh8g7" pod "3afca0a5-ee7e-4c95-80b6-441fea71ade3" (UID: "3afca0a5-ee7e-4c95-80b6-441fea71ade3"). InnerVolumeSpecName "kube-api-access-hh8g7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:58:23.323657 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.323636    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-data-science-collector-ta-client-cert" pod "3afca0a5-ee7e-4c95-80b6-441fea71ade3" (UID: "3afca0a5-ee7e-4c95-80b6-441fea71ade3"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:23.323827 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.323802    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-tls-certs" pod "3afca0a5-ee7e-4c95-80b6-441fea71ade3" (UID: "3afca0a5-ee7e-4c95-80b6-441fea71ade3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:23.422572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.422511    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:23.422572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.422535    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:23.422572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.422549    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh8g7\" (UniqueName: \"kubernetes.io/projected/3afca0a5-ee7e-4c95-80b6-441fea71ade3-kube-api-access-hh8g7\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:23.422572 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.422561    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/3afca0a5-ee7e-4c95-80b6-441fea71ade3-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:23.600106 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.600075    4849 generic.go:356] "Generic (PLEG): container finished" podID="3afca0a5-ee7e-4c95-80b6-441fea71ade3" containerID="c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105" exitCode=0
Oct 30 10:58:23.600218 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.600129    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.600218 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.600160    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"3afca0a5-ee7e-4c95-80b6-441fea71ade3","Type":"ContainerDied","Data":"c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105"}
Oct 30 10:58:23.600218 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.600216    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"3afca0a5-ee7e-4c95-80b6-441fea71ade3","Type":"ContainerDied","Data":"2fee28dd528f0e44e882b92d6d2642febdf3174214ae95d7c0e29a503999f715"}
Oct 30 10:58:23.600325 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.600239    4849 scope.go:122] "RemoveContainer" containerID="c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105"
Oct 30 10:58:23.609405 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.609386    4849 scope.go:122] "RemoveContainer" containerID="c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105"
Oct 30 10:58:23.609688 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:58:23.609665    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105\": container with ID starting with c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105 not found: ID does not exist" containerID="c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105"
Oct 30 10:58:23.609743 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.609696    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105"} err="failed to get container status \"c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105\": rpc error: code = NotFound desc = could not find container \"c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105\": container with ID starting with c5e2c992116568c2e4d57620321454e2cf0a02c7515242ff969c353e9d153105 not found: ID does not exist"
Oct 30 10:58:23.654783 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.654760    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:23.660398 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.660373    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:23.695655 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.695599    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:23.695981 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.695968    4849 cpu_manager.go:414] "containerMap: removing container" podUID="3afca0a5-ee7e-4c95-80b6-441fea71ade3" containerName="otc-container"
Oct 30 10:58:23.696045 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.695983    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3afca0a5-ee7e-4c95-80b6-441fea71ade3" containerName="otc-container"
Oct 30 10:58:23.696089 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.696080    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="3afca0a5-ee7e-4c95-80b6-441fea71ade3" containerName="otc-container"
Oct 30 10:58:23.700682 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.700667    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.718569 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.718549    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:58:23.718668 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.718624    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:58:23.718668 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.718627    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:58:23.718767 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.718749    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:58:23.720971 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.720949    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:23.725071 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.725051    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.725152 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.725089    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9200b631-f8d7-4eff-acf1-264d4e0d1c62-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.725209 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.725197    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.725254 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.725239    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.725298 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.725256    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnv6d\" (UniqueName: \"kubernetes.io/projected/9200b631-f8d7-4eff-acf1-264d4e0d1c62-kube-api-access-pnv6d\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.813261 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.813237    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:58:23.826578 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.826554    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.826681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.826580    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnv6d\" (UniqueName: \"kubernetes.io/projected/9200b631-f8d7-4eff-acf1-264d4e0d1c62-kube-api-access-pnv6d\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.826681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.826600    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.826681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.826624    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9200b631-f8d7-4eff-acf1-264d4e0d1c62-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.826681 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.826671    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.827332 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.827309    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9200b631-f8d7-4eff-acf1-264d4e0d1c62-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.828970 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.828952    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.829095 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.829078    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.829143 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.829084    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:23.850585 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:23.850559    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnv6d\" (UniqueName: \"kubernetes.io/projected/9200b631-f8d7-4eff-acf1-264d4e0d1c62-kube-api-access-pnv6d\") pod \"data-science-collector-collector-0\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:24.013530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.013475    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:24.313435 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.313407    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:24.314929 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:58:24.314905    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9200b631_f8d7_4eff_acf1_264d4e0d1c62.slice/crio-3d9efecf6fed7d5df3064ef835e6e1f61f1c07433301d06cb6afd3cb799a4c9b WatchSource:0}: Error finding container 3d9efecf6fed7d5df3064ef835e6e1f61f1c07433301d06cb6afd3cb799a4c9b: Status 404 returned error can't find the container with id 3d9efecf6fed7d5df3064ef835e6e1f61f1c07433301d06cb6afd3cb799a4c9b
Oct 30 10:58:24.317079 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.317050    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:58:24.317157 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.317111    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:58:24.608775 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.608694    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9200b631-f8d7-4eff-acf1-264d4e0d1c62","Type":"ContainerStarted","Data":"6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e"}
Oct 30 10:58:24.608775 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.608743    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9200b631-f8d7-4eff-acf1-264d4e0d1c62","Type":"ContainerStarted","Data":"3d9efecf6fed7d5df3064ef835e6e1f61f1c07433301d06cb6afd3cb799a4c9b"}
Oct 30 10:58:24.653477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.653429    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.653410568 podStartE2EDuration="1.653410568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:58:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:58:24.651962768 +0000 UTC m=+802.032412208" watchObservedRunningTime="2026-10-30 10:58:24.653410568 +0000 UTC m=+802.033860009"
Oct 30 10:58:24.754212 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:24.754180    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3afca0a5-ee7e-4c95-80b6-441fea71ade3" path="/var/lib/kubelet/pods/3afca0a5-ee7e-4c95-80b6-441fea71ade3/volumes"
Oct 30 10:58:33.099979 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.099941    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:33.100560 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.100496    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="9200b631-f8d7-4eff-acf1-264d4e0d1c62" containerName="otc-container" containerID="cri-o://6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e" gracePeriod=30
Oct 30 10:58:33.350388 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.350336    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:33.400187 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.400156    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9200b631-f8d7-4eff-acf1-264d4e0d1c62-otc-internal\" (UniqueName: \"kubernetes.io/configmap/9200b631-f8d7-4eff-acf1-264d4e0d1c62-otc-internal\") pod \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") "
Oct 30 10:58:33.400312 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.400215    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-monitoring-tls-certs\") pod \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") "
Oct 30 10:58:33.400312 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.400256    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-tls-certs\") pod \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") "
Oct 30 10:58:33.400312 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.400306    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-data-science-collector-ta-client-cert\") pod \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") "
Oct 30 10:58:33.400485 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.400394    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9200b631-f8d7-4eff-acf1-264d4e0d1c62-kube-api-access-pnv6d\" (UniqueName: \"kubernetes.io/projected/9200b631-f8d7-4eff-acf1-264d4e0d1c62-kube-api-access-pnv6d\") pod \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\" (UID: \"9200b631-f8d7-4eff-acf1-264d4e0d1c62\") "
Oct 30 10:58:33.400879 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.400849    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9200b631-f8d7-4eff-acf1-264d4e0d1c62-otc-internal" pod "9200b631-f8d7-4eff-acf1-264d4e0d1c62" (UID: "9200b631-f8d7-4eff-acf1-264d4e0d1c62"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 10:58:33.402858 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.402828    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-monitoring-tls-certs" pod "9200b631-f8d7-4eff-acf1-264d4e0d1c62" (UID: "9200b631-f8d7-4eff-acf1-264d4e0d1c62"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:33.403912 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.403426    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-data-science-collector-ta-client-cert" pod "9200b631-f8d7-4eff-acf1-264d4e0d1c62" (UID: "9200b631-f8d7-4eff-acf1-264d4e0d1c62"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:33.403912 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.403549    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-tls-certs" pod "9200b631-f8d7-4eff-acf1-264d4e0d1c62" (UID: "9200b631-f8d7-4eff-acf1-264d4e0d1c62"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 10:58:33.404275 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.404250    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9200b631-f8d7-4eff-acf1-264d4e0d1c62-kube-api-access-pnv6d" pod "9200b631-f8d7-4eff-acf1-264d4e0d1c62" (UID: "9200b631-f8d7-4eff-acf1-264d4e0d1c62"). InnerVolumeSpecName "kube-api-access-pnv6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:58:33.501511 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.501482    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/9200b631-f8d7-4eff-acf1-264d4e0d1c62-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:33.501511 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.501508    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:33.501675 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.501546    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:33.501675 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.501561    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/9200b631-f8d7-4eff-acf1-264d4e0d1c62-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:33.501675 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.501574    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnv6d\" (UniqueName: \"kubernetes.io/projected/9200b631-f8d7-4eff-acf1-264d4e0d1c62-kube-api-access-pnv6d\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:58:33.564360 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.564331    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"]
Oct 30 10:58:33.564699 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.564685    4849 cpu_manager.go:414] "containerMap: removing container" podUID="9200b631-f8d7-4eff-acf1-264d4e0d1c62" containerName="otc-container"
Oct 30 10:58:33.564699 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.564699    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9200b631-f8d7-4eff-acf1-264d4e0d1c62" containerName="otc-container"
Oct 30 10:58:33.564848 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.564772    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="9200b631-f8d7-4eff-acf1-264d4e0d1c62" containerName="otc-container"
Oct 30 10:58:33.567986 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.567970    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.585016 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.584994    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"tests-monitoring-tenant-metrics\"/\"default-dockercfg-ww85p\""
Oct 30 10:58:33.585144 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.584994    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"tests-monitoring-tenant-metrics\"/\"openshift-service-ca.crt\""
Oct 30 10:58:33.585144 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.585052    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"tests-monitoring-tenant-metrics\"/\"kube-root-ca.crt\""
Oct 30 10:58:33.586959 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.586939    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"]
Oct 30 10:58:33.602554 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.602508    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g86k9\" (UniqueName: \"kubernetes.io/projected/70e79ad0-bba4-4f05-aee1-e1328171df70-kube-api-access-g86k9\") pod \"tenant-metrics-exporter-5cf544fb97-kx9vd\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.602554 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.602535    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70e79ad0-bba4-4f05-aee1-e1328171df70-data\") pod \"tenant-metrics-exporter-5cf544fb97-kx9vd\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.665229 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.665203    4849 generic.go:356] "Generic (PLEG): container finished" podID="9200b631-f8d7-4eff-acf1-264d4e0d1c62" containerID="6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e" exitCode=0
Oct 30 10:58:33.665341 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.665244    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:33.665341 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.665291    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9200b631-f8d7-4eff-acf1-264d4e0d1c62","Type":"ContainerDied","Data":"6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e"}
Oct 30 10:58:33.665432 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.665349    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"9200b631-f8d7-4eff-acf1-264d4e0d1c62","Type":"ContainerDied","Data":"3d9efecf6fed7d5df3064ef835e6e1f61f1c07433301d06cb6afd3cb799a4c9b"}
Oct 30 10:58:33.665432 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.665379    4849 scope.go:122] "RemoveContainer" containerID="6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e"
Oct 30 10:58:33.675807 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.675793    4849 scope.go:122] "RemoveContainer" containerID="6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e"
Oct 30 10:58:33.676091 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:58:33.676073    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e\": container with ID starting with 6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e not found: ID does not exist" containerID="6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e"
Oct 30 10:58:33.676162 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.676102    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e"} err="failed to get container status \"6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e\": rpc error: code = NotFound desc = could not find container \"6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e\": container with ID starting with 6fadd0f0f336c90903dd8d2c78d723018acd2cad62128721d7949cb3ca66bd3e not found: ID does not exist"
Oct 30 10:58:33.703658 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.703636    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g86k9\" (UniqueName: \"kubernetes.io/projected/70e79ad0-bba4-4f05-aee1-e1328171df70-kube-api-access-g86k9\") pod \"tenant-metrics-exporter-5cf544fb97-kx9vd\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.703750 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.703668    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70e79ad0-bba4-4f05-aee1-e1328171df70-data\") pod \"tenant-metrics-exporter-5cf544fb97-kx9vd\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.704086 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.704071    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70e79ad0-bba4-4f05-aee1-e1328171df70-data\") pod \"tenant-metrics-exporter-5cf544fb97-kx9vd\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.722868 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.722848    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:33.726588 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.726566    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:33.727772 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.727753    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g86k9\" (UniqueName: \"kubernetes.io/projected/70e79ad0-bba4-4f05-aee1-e1328171df70-kube-api-access-g86k9\") pod \"tenant-metrics-exporter-5cf544fb97-kx9vd\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.774549 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.774516    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:33.779372 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.779357    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:33.797342 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.797234    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 10:58:33.798252 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.798235    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 10:58:33.798367 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.798299    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 10:58:33.798367 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.798324    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 10:58:33.798464 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.798378    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 10:58:33.799167 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.799146    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:33.881358 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.881302    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:58:33.905015 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.904993    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f39c31bd-2359-47b8-ab9f-39474113fe61-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:33.905151 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.905128    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:33.905207 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.905193    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:33.905244 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.905225    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:33.905275 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:33.905254    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-455fl\" (UniqueName: \"kubernetes.io/projected/f39c31bd-2359-47b8-ab9f-39474113fe61-kube-api-access-455fl\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.006687 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.006656    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.006838 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.006697    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.006838 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.006728    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.006838 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.006755    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-455fl\" (UniqueName: \"kubernetes.io/projected/f39c31bd-2359-47b8-ab9f-39474113fe61-kube-api-access-455fl\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.006838 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.006776    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f39c31bd-2359-47b8-ab9f-39474113fe61-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.007537 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.007511    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f39c31bd-2359-47b8-ab9f-39474113fe61-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.009120 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.009101    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.009120 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.009118    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.009231 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.009217    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.030936 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.030907    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-455fl\" (UniqueName: \"kubernetes.io/projected/f39c31bd-2359-47b8-ab9f-39474113fe61-kube-api-access-455fl\") pod \"data-science-collector-collector-0\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.091904 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.091878    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 10:58:34.181179 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.181154    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"]
Oct 30 10:58:34.183143 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:58:34.183118    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e79ad0_bba4_4f05_aee1_e1328171df70.slice/crio-9fbef981e6994e4d1146cf49416a6721eecc41475888fd203a62b57a37f95410 WatchSource:0}: Error finding container 9fbef981e6994e4d1146cf49416a6721eecc41475888fd203a62b57a37f95410: Status 404 returned error can't find the container with id 9fbef981e6994e4d1146cf49416a6721eecc41475888fd203a62b57a37f95410
Oct 30 10:58:34.185226 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.185206    4849 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 10:58:34.397627 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.397607    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 10:58:34.399935 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:58:34.399906    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39c31bd_2359_47b8_ab9f_39474113fe61.slice/crio-3a704d2729ac0a6d014da0bfe9b531ad843c683be49cd3480bbccf321c5347cf WatchSource:0}: Error finding container 3a704d2729ac0a6d014da0bfe9b531ad843c683be49cd3480bbccf321c5347cf: Status 404 returned error can't find the container with id 3a704d2729ac0a6d014da0bfe9b531ad843c683be49cd3480bbccf321c5347cf
Oct 30 10:58:34.402094 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.402063    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:58:34.402185 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.402119    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 10:58:34.675301 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.675221    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f39c31bd-2359-47b8-ab9f-39474113fe61","Type":"ContainerStarted","Data":"e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0"}
Oct 30 10:58:34.675301 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.675286    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f39c31bd-2359-47b8-ab9f-39474113fe61","Type":"ContainerStarted","Data":"3a704d2729ac0a6d014da0bfe9b531ad843c683be49cd3480bbccf321c5347cf"}
Oct 30 10:58:34.677937 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.677907    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd" event={"ID":"70e79ad0-bba4-4f05-aee1-e1328171df70","Type":"ContainerStarted","Data":"9fbef981e6994e4d1146cf49416a6721eecc41475888fd203a62b57a37f95410"}
Oct 30 10:58:34.720686 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.720591    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.7205732230000002 podStartE2EDuration="1.720573223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:58:34.720231941 +0000 UTC m=+812.100681381" watchObservedRunningTime="2026-10-30 10:58:34.720573223 +0000 UTC m=+812.101022663"
Oct 30 10:58:34.758727 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:34.758695    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9200b631-f8d7-4eff-acf1-264d4e0d1c62" path="/var/lib/kubelet/pods/9200b631-f8d7-4eff-acf1-264d4e0d1c62/volumes"
Oct 30 10:58:38.711150 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:38.711110    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd" event={"ID":"70e79ad0-bba4-4f05-aee1-e1328171df70","Type":"ContainerStarted","Data":"4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3"}
Oct 30 10:58:38.752808 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:58:38.752744    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd" podStartSLOduration=2.182391397 podStartE2EDuration="5.752726715s" totalImagesPullingTime="3.570335318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:58:38.752165433 +0000 UTC m=+816.132614890" watchObservedRunningTime="2026-10-30 10:58:38.752726715 +0000 UTC m=+816.133176155"
Oct 30 10:58:39.430403 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:58:39.430367    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:59:09.433516 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:59:09.433474    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:59:39.430683 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:59:39.430642    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 10:59:40.362646 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.362609    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"]
Oct 30 10:59:40.364808 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.362818    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd" podUID="70e79ad0-bba4-4f05-aee1-e1328171df70" containerName="tenant-metrics-exporter" containerID="cri-o://4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3" gracePeriod=30
Oct 30 10:59:40.717010 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.716989    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:59:40.721029 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.721000    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70e79ad0-bba4-4f05-aee1-e1328171df70-kube-api-access-g86k9\" (UniqueName: \"kubernetes.io/projected/70e79ad0-bba4-4f05-aee1-e1328171df70-kube-api-access-g86k9\") pod \"70e79ad0-bba4-4f05-aee1-e1328171df70\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") "
Oct 30 10:59:40.722913 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.722890    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e79ad0-bba4-4f05-aee1-e1328171df70-kube-api-access-g86k9" pod "70e79ad0-bba4-4f05-aee1-e1328171df70" (UID: "70e79ad0-bba4-4f05-aee1-e1328171df70"). InnerVolumeSpecName "kube-api-access-g86k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 10:59:40.821474 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.821453    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70e79ad0-bba4-4f05-aee1-e1328171df70-data\" (UniqueName: \"kubernetes.io/empty-dir/70e79ad0-bba4-4f05-aee1-e1328171df70-data\") pod \"70e79ad0-bba4-4f05-aee1-e1328171df70\" (UID: \"70e79ad0-bba4-4f05-aee1-e1328171df70\") "
Oct 30 10:59:40.821831 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.821811    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g86k9\" (UniqueName: \"kubernetes.io/projected/70e79ad0-bba4-4f05-aee1-e1328171df70-kube-api-access-g86k9\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:59:40.821902 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.821840    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e79ad0-bba4-4f05-aee1-e1328171df70-data" pod "70e79ad0-bba4-4f05-aee1-e1328171df70" (UID: "70e79ad0-bba4-4f05-aee1-e1328171df70"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 10:59:40.922609 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:40.922558    4849 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70e79ad0-bba4-4f05-aee1-e1328171df70-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 10:59:41.045640 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.045609    4849 generic.go:356] "Generic (PLEG): container finished" podID="70e79ad0-bba4-4f05-aee1-e1328171df70" containerID="4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3" exitCode=0
Oct 30 10:59:41.045731 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.045658    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"
Oct 30 10:59:41.045731 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.045695    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd" event={"ID":"70e79ad0-bba4-4f05-aee1-e1328171df70","Type":"ContainerDied","Data":"4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3"}
Oct 30 10:59:41.045807 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.045750    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd" event={"ID":"70e79ad0-bba4-4f05-aee1-e1328171df70","Type":"ContainerDied","Data":"9fbef981e6994e4d1146cf49416a6721eecc41475888fd203a62b57a37f95410"}
Oct 30 10:59:41.045807 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.045772    4849 scope.go:122] "RemoveContainer" containerID="4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3"
Oct 30 10:59:41.054798 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.054777    4849 scope.go:122] "RemoveContainer" containerID="4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3"
Oct 30 10:59:41.055103 ip-10-0-140-230 kubenswrapper[4849]: E1030 10:59:41.055084    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3\": container with ID starting with 4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3 not found: ID does not exist" containerID="4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3"
Oct 30 10:59:41.055162 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.055118    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3"} err="failed to get container status \"4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3\": rpc error: code = NotFound desc = could not find container \"4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3\": container with ID starting with 4c2404c19873d1eccf317834bb707e85214b332d9a34f83af72d084539e154b3 not found: ID does not exist"
Oct 30 10:59:41.100240 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.100218    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"]
Oct 30 10:59:41.113616 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:41.113597    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-kx9vd"]
Oct 30 10:59:42.753695 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:42.753657    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70e79ad0-bba4-4f05-aee1-e1328171df70" path="/var/lib/kubelet/pods/70e79ad0-bba4-4f05-aee1-e1328171df70/volumes"
Oct 30 10:59:52.807131 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.807097    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"]
Oct 30 10:59:52.807496 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.807444    4849 cpu_manager.go:414] "containerMap: removing container" podUID="70e79ad0-bba4-4f05-aee1-e1328171df70" containerName="tenant-metrics-exporter"
Oct 30 10:59:52.807496 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.807454    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70e79ad0-bba4-4f05-aee1-e1328171df70" containerName="tenant-metrics-exporter"
Oct 30 10:59:52.807566 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.807525    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="70e79ad0-bba4-4f05-aee1-e1328171df70" containerName="tenant-metrics-exporter"
Oct 30 10:59:52.810492 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.810469    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:52.830305 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.830276    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"tests-monitoring-dcgm-https-18e34997bcb25330\"/\"dcgm-https-exporter\""
Oct 30 10:59:52.830448 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.830368    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"]
Oct 30 10:59:52.841267 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.841247    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"tests-monitoring-dcgm-https-18e34997bcb25330\"/\"dcgm-https-tls\""
Oct 30 10:59:52.841394 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.841295    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"tests-monitoring-dcgm-https-18e34997bcb25330\"/\"default-dockercfg-4zf4p\""
Oct 30 10:59:52.912864 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.912841    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6906bc04-cba4-40e3-87d0-56445f5fee33-data\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:52.913010 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.912871    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/6906bc04-cba4-40e3-87d0-56445f5fee33-tls\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:52.913116 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:52.913074    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/configmap/6906bc04-cba4-40e3-87d0-56445f5fee33-web-config\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.013882 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.013861    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/configmap/6906bc04-cba4-40e3-87d0-56445f5fee33-web-config\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.013988 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.013894    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6906bc04-cba4-40e3-87d0-56445f5fee33-data\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.013988 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.013910    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/6906bc04-cba4-40e3-87d0-56445f5fee33-tls\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.014341 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.014319    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6906bc04-cba4-40e3-87d0-56445f5fee33-data\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.014530 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.014511    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/configmap/6906bc04-cba4-40e3-87d0-56445f5fee33-web-config\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.016477 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.016454    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls\" (UniqueName: \"kubernetes.io/secret/6906bc04-cba4-40e3-87d0-56445f5fee33-tls\") pod \"dcgm-https-exporter\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") " pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.123236 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.123179    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:53.421229 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:53.421182    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"]
Oct 30 10:59:53.423510 ip-10-0-140-230 kubenswrapper[4849]: W1030 10:59:53.423481    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6906bc04_cba4_40e3_87d0_56445f5fee33.slice/crio-ae98c57e653aefd1e32b476119ccdbec22cfb64b8a2784a3dba7e2aa80358211 WatchSource:0}: Error finding container ae98c57e653aefd1e32b476119ccdbec22cfb64b8a2784a3dba7e2aa80358211: Status 404 returned error can't find the container with id ae98c57e653aefd1e32b476119ccdbec22cfb64b8a2784a3dba7e2aa80358211
Oct 30 10:59:54.115722 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:54.115683    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter" event={"ID":"6906bc04-cba4-40e3-87d0-56445f5fee33","Type":"ContainerStarted","Data":"ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953"}
Oct 30 10:59:54.116140 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:54.115786    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 10:59:54.116140 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:54.115818    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter" event={"ID":"6906bc04-cba4-40e3-87d0-56445f5fee33","Type":"ContainerStarted","Data":"ae98c57e653aefd1e32b476119ccdbec22cfb64b8a2784a3dba7e2aa80358211"}
Oct 30 10:59:54.162830 ip-10-0-140-230 kubenswrapper[4849]: I1030 10:59:54.162785    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter" podStartSLOduration=2.162772516 podStartE2EDuration="2.162772516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 10:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 10:59:54.160404093 +0000 UTC m=+891.540853532" watchObservedRunningTime="2026-10-30 10:59:54.162772516 +0000 UTC m=+891.543221954"
Oct 30 11:00:00.123194 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:00.123165    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 11:00:02.842842 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:02.842809    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:00:02.845628 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:02.845607    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:00:09.369798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:09.369758    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:00:09.370662 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:09.370595    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="f39c31bd-2359-47b8-ab9f-39474113fe61" containerName="otc-container" containerID="cri-o://e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0" gracePeriod=30
Oct 30 11:00:09.432077 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:00:09.432042    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:00:14.624504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.624480    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:14.677776 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.677750    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f39c31bd-2359-47b8-ab9f-39474113fe61-kube-api-access-455fl\" (UniqueName: \"kubernetes.io/projected/f39c31bd-2359-47b8-ab9f-39474113fe61-kube-api-access-455fl\") pod \"f39c31bd-2359-47b8-ab9f-39474113fe61\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") "
Oct 30 11:00:14.677922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.677796    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-data-science-collector-ta-client-cert\") pod \"f39c31bd-2359-47b8-ab9f-39474113fe61\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") "
Oct 30 11:00:14.677922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.677842    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-tls-certs\") pod \"f39c31bd-2359-47b8-ab9f-39474113fe61\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") "
Oct 30 11:00:14.678063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.677949    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f39c31bd-2359-47b8-ab9f-39474113fe61-otc-internal\" (UniqueName: \"kubernetes.io/configmap/f39c31bd-2359-47b8-ab9f-39474113fe61-otc-internal\") pod \"f39c31bd-2359-47b8-ab9f-39474113fe61\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") "
Oct 30 11:00:14.678063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.677982    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-monitoring-tls-certs\") pod \"f39c31bd-2359-47b8-ab9f-39474113fe61\" (UID: \"f39c31bd-2359-47b8-ab9f-39474113fe61\") "
Oct 30 11:00:14.678644 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.678602    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39c31bd-2359-47b8-ab9f-39474113fe61-otc-internal" pod "f39c31bd-2359-47b8-ab9f-39474113fe61" (UID: "f39c31bd-2359-47b8-ab9f-39474113fe61"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:00:14.680371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.680333    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-data-science-collector-ta-client-cert" pod "f39c31bd-2359-47b8-ab9f-39474113fe61" (UID: "f39c31bd-2359-47b8-ab9f-39474113fe61"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:00:14.680533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.680496    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39c31bd-2359-47b8-ab9f-39474113fe61-kube-api-access-455fl" pod "f39c31bd-2359-47b8-ab9f-39474113fe61" (UID: "f39c31bd-2359-47b8-ab9f-39474113fe61"). InnerVolumeSpecName "kube-api-access-455fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:00:14.680611 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.680574    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-tls-certs" pod "f39c31bd-2359-47b8-ab9f-39474113fe61" (UID: "f39c31bd-2359-47b8-ab9f-39474113fe61"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:00:14.680892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.680866    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-monitoring-tls-certs" pod "f39c31bd-2359-47b8-ab9f-39474113fe61" (UID: "f39c31bd-2359-47b8-ab9f-39474113fe61"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:00:14.778862 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.778838    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f39c31bd-2359-47b8-ab9f-39474113fe61-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:00:14.778862 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.778861    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:00:14.778991 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.778871    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-455fl\" (UniqueName: \"kubernetes.io/projected/f39c31bd-2359-47b8-ab9f-39474113fe61-kube-api-access-455fl\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:00:14.778991 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.778881    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:00:14.778991 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:14.778890    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f39c31bd-2359-47b8-ab9f-39474113fe61-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:00:15.227709 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.227681    4849 generic.go:356] "Generic (PLEG): container finished" podID="f39c31bd-2359-47b8-ab9f-39474113fe61" containerID="e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0" exitCode=0
Oct 30 11:00:15.227844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.227725    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.227844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.227737    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f39c31bd-2359-47b8-ab9f-39474113fe61","Type":"ContainerDied","Data":"e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0"}
Oct 30 11:00:15.227844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.227775    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f39c31bd-2359-47b8-ab9f-39474113fe61","Type":"ContainerDied","Data":"3a704d2729ac0a6d014da0bfe9b531ad843c683be49cd3480bbccf321c5347cf"}
Oct 30 11:00:15.227844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.227797    4849 scope.go:122] "RemoveContainer" containerID="e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0"
Oct 30 11:00:15.236430 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.236408    4849 scope.go:122] "RemoveContainer" containerID="e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0"
Oct 30 11:00:15.236713 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:00:15.236689    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0\": container with ID starting with e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0 not found: ID does not exist" containerID="e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0"
Oct 30 11:00:15.236781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.236725    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0"} err="failed to get container status \"e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0\": rpc error: code = NotFound desc = could not find container \"e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0\": container with ID starting with e4d9d5ecd7f54514c4412a7ca69bd7236c249769040e362b673b422ff869b1e0 not found: ID does not exist"
Oct 30 11:00:15.283056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.283032    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:00:15.296504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.296482    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:00:15.337892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.337861    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:00:15.338266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.338249    4849 cpu_manager.go:414] "containerMap: removing container" podUID="f39c31bd-2359-47b8-ab9f-39474113fe61" containerName="otc-container"
Oct 30 11:00:15.338266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.338266    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f39c31bd-2359-47b8-ab9f-39474113fe61" containerName="otc-container"
Oct 30 11:00:15.338403 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.338347    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="f39c31bd-2359-47b8-ab9f-39474113fe61" containerName="otc-container"
Oct 30 11:00:15.342861 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.342844    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.361774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.361755    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:00:15.361886 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.361787    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 11:00:15.361886 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.361820    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:00:15.362003 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.361918    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:00:15.362801 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.362779    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-99980381\""
Oct 30 11:00:15.364186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.364169    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:00:15.381186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.381168    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.381264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.381196    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6c759452-49a0-4a19-a9e6-8eae97e084ca-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.381315 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.381293    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9m2c\" (UniqueName: \"kubernetes.io/projected/6c759452-49a0-4a19-a9e6-8eae97e084ca-kube-api-access-r9m2c\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.381368 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.381356    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.381401 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.381378    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.482229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.482171    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9m2c\" (UniqueName: \"kubernetes.io/projected/6c759452-49a0-4a19-a9e6-8eae97e084ca-kube-api-access-r9m2c\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.482229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.482206    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.482229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.482224    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.482489 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.482467    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.482551 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.482517    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6c759452-49a0-4a19-a9e6-8eae97e084ca-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.483186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.483166    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6c759452-49a0-4a19-a9e6-8eae97e084ca-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.484844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.484821    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.484946 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.484886    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.484946 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.484895    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.516802 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.516778    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9m2c\" (UniqueName: \"kubernetes.io/projected/6c759452-49a0-4a19-a9e6-8eae97e084ca-kube-api-access-r9m2c\") pod \"data-science-collector-collector-0\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.655360 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.655327    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:00:15.954928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.954904    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:00:15.957164 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:00:15.957135    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c759452_49a0_4a19_a9e6_8eae97e084ca.slice/crio-dddf1e6ef374ab15b5c3f4959f84e3c11d2c30e72f88b45e4b5b47440b200f8b WatchSource:0}: Error finding container dddf1e6ef374ab15b5c3f4959f84e3c11d2c30e72f88b45e4b5b47440b200f8b: Status 404 returned error can't find the container with id dddf1e6ef374ab15b5c3f4959f84e3c11d2c30e72f88b45e4b5b47440b200f8b
Oct 30 11:00:15.959289 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.959262    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:00:15.959364 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:15.959315    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:00:16.240535 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:16.240448    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6c759452-49a0-4a19-a9e6-8eae97e084ca","Type":"ContainerStarted","Data":"651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587"}
Oct 30 11:00:16.240701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:16.240544    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6c759452-49a0-4a19-a9e6-8eae97e084ca","Type":"ContainerStarted","Data":"dddf1e6ef374ab15b5c3f4959f84e3c11d2c30e72f88b45e4b5b47440b200f8b"}
Oct 30 11:00:16.285456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:16.285406    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.285393351 podStartE2EDuration="1.285393351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:00:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:00:16.283783636 +0000 UTC m=+913.664233105" watchObservedRunningTime="2026-10-30 11:00:16.285393351 +0000 UTC m=+913.665842844"
Oct 30 11:00:16.753606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:00:16.753572    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f39c31bd-2359-47b8-ab9f-39474113fe61" path="/var/lib/kubelet/pods/f39c31bd-2359-47b8-ab9f-39474113fe61/volumes"
Oct 30 11:00:39.430439 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:00:39.430405    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:01:02.907389 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:02.907359    4849 scope.go:122] "RemoveContainer" containerID="bed8d7be25f214e9d434c8fb4d1b973b9cb37b8c4c00b87920eeb7debbd91bc7"
Oct 30 11:01:02.915570 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:02.915550    4849 scope.go:122] "RemoveContainer" containerID="7c497d2458a7ab7d218512fc08ca1206f71358b396d4b578c7f9b18a1169c767"
Oct 30 11:01:02.923000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:02.922982    4849 scope.go:122] "RemoveContainer" containerID="2eba01aff7128ff490deb051e16afd154b7f0642413c0335af5854580fe493eb"
Oct 30 11:01:09.432548 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:01:09.432508    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:01:39.433476 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:01:39.433430    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:01:45.757650 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:45.757610    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:01:45.758726 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:45.758085    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="6c759452-49a0-4a19-a9e6-8eae97e084ca" containerName="otc-container" containerID="cri-o://651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587" gracePeriod=30
Oct 30 11:01:48.105122 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.105094    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:48.195373 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.195308    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-tls-certs\") pod \"6c759452-49a0-4a19-a9e6-8eae97e084ca\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") "
Oct 30 11:01:48.195485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.195399    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-data-science-collector-ta-client-cert\") pod \"6c759452-49a0-4a19-a9e6-8eae97e084ca\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") "
Oct 30 11:01:48.195485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.195421    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c759452-49a0-4a19-a9e6-8eae97e084ca-otc-internal\" (UniqueName: \"kubernetes.io/configmap/6c759452-49a0-4a19-a9e6-8eae97e084ca-otc-internal\") pod \"6c759452-49a0-4a19-a9e6-8eae97e084ca\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") "
Oct 30 11:01:48.195485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.195440    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-monitoring-tls-certs\") pod \"6c759452-49a0-4a19-a9e6-8eae97e084ca\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") "
Oct 30 11:01:48.195614 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.195514    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c759452-49a0-4a19-a9e6-8eae97e084ca-kube-api-access-r9m2c\" (UniqueName: \"kubernetes.io/projected/6c759452-49a0-4a19-a9e6-8eae97e084ca-kube-api-access-r9m2c\") pod \"6c759452-49a0-4a19-a9e6-8eae97e084ca\" (UID: \"6c759452-49a0-4a19-a9e6-8eae97e084ca\") "
Oct 30 11:01:48.195837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.195811    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c759452-49a0-4a19-a9e6-8eae97e084ca-otc-internal" pod "6c759452-49a0-4a19-a9e6-8eae97e084ca" (UID: "6c759452-49a0-4a19-a9e6-8eae97e084ca"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:01:48.197434 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.197403    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-tls-certs" pod "6c759452-49a0-4a19-a9e6-8eae97e084ca" (UID: "6c759452-49a0-4a19-a9e6-8eae97e084ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:01:48.197546 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.197521    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c759452-49a0-4a19-a9e6-8eae97e084ca-kube-api-access-r9m2c" pod "6c759452-49a0-4a19-a9e6-8eae97e084ca" (UID: "6c759452-49a0-4a19-a9e6-8eae97e084ca"). InnerVolumeSpecName "kube-api-access-r9m2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:01:48.197894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.197872    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-data-science-collector-ta-client-cert" pod "6c759452-49a0-4a19-a9e6-8eae97e084ca" (UID: "6c759452-49a0-4a19-a9e6-8eae97e084ca"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:01:48.198004 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.197895    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-monitoring-tls-certs" pod "6c759452-49a0-4a19-a9e6-8eae97e084ca" (UID: "6c759452-49a0-4a19-a9e6-8eae97e084ca"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:01:48.297094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.297071    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:48.297094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.297094    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:48.297234 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.297104    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6c759452-49a0-4a19-a9e6-8eae97e084ca-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:48.297234 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.297113    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c759452-49a0-4a19-a9e6-8eae97e084ca-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:48.297234 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.297122    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9m2c\" (UniqueName: \"kubernetes.io/projected/6c759452-49a0-4a19-a9e6-8eae97e084ca-kube-api-access-r9m2c\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:48.729906 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.729875    4849 generic.go:356] "Generic (PLEG): container finished" podID="6c759452-49a0-4a19-a9e6-8eae97e084ca" containerID="651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587" exitCode=0
Oct 30 11:01:48.730101 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.729919    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:48.730101 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.729965    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6c759452-49a0-4a19-a9e6-8eae97e084ca","Type":"ContainerDied","Data":"651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587"}
Oct 30 11:01:48.730101 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.730045    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6c759452-49a0-4a19-a9e6-8eae97e084ca","Type":"ContainerDied","Data":"dddf1e6ef374ab15b5c3f4959f84e3c11d2c30e72f88b45e4b5b47440b200f8b"}
Oct 30 11:01:48.730101 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.730071    4849 scope.go:122] "RemoveContainer" containerID="651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587"
Oct 30 11:01:48.739384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.739367    4849 scope.go:122] "RemoveContainer" containerID="651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587"
Oct 30 11:01:48.739652 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:01:48.739634    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587\": container with ID starting with 651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587 not found: ID does not exist" containerID="651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587"
Oct 30 11:01:48.739705 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.739660    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587"} err="failed to get container status \"651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587\": rpc error: code = NotFound desc = could not find container \"651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587\": container with ID starting with 651144566b4f564383c8936dc53d420a84b7e119454da594009eefee39af6587 not found: ID does not exist"
Oct 30 11:01:48.784344 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.784321    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:01:48.797857 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.797835    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:01:48.822410 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.822391    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:01:48.822746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.822733    4849 cpu_manager.go:414] "containerMap: removing container" podUID="6c759452-49a0-4a19-a9e6-8eae97e084ca" containerName="otc-container"
Oct 30 11:01:48.822793 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.822746    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c759452-49a0-4a19-a9e6-8eae97e084ca" containerName="otc-container"
Oct 30 11:01:48.822834 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.822798    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c759452-49a0-4a19-a9e6-8eae97e084ca" containerName="otc-container"
Oct 30 11:01:48.830011 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.829996    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:48.847069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.846886    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:01:48.847069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.847009    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 11:01:48.847069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.847007    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:01:48.847285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.847129    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:01:48.847425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.847402    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:01:48.847867 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:48.847849    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:01:49.003115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.003056    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgs9v\" (UniqueName: \"kubernetes.io/projected/e8a878c7-736f-42c8-a764-e4c3fddddeb8-kube-api-access-zgs9v\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.003115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.003111    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.003260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.003183    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.003260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.003209    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8a878c7-736f-42c8-a764-e4c3fddddeb8-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.003260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.003232    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.084940 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.084915    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"]
Oct 30 11:01:49.085107 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.085090    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter" podUID="6906bc04-cba4-40e3-87d0-56445f5fee33" containerName="dcgm-https-exporter" containerID="cri-o://ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953" gracePeriod=30
Oct 30 11:01:49.104274 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.104251    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.104387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.104296    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.104387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.104320    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8a878c7-736f-42c8-a764-e4c3fddddeb8-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.104387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.104353    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.104550 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.104385    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgs9v\" (UniqueName: \"kubernetes.io/projected/e8a878c7-736f-42c8-a764-e4c3fddddeb8-kube-api-access-zgs9v\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.105165 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.105143    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8a878c7-736f-42c8-a764-e4c3fddddeb8-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.106815 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.106794    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.106815 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.106794    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.106960 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.106836    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.128773 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.128745    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgs9v\" (UniqueName: \"kubernetes.io/projected/e8a878c7-736f-42c8-a764-e4c3fddddeb8-kube-api-access-zgs9v\") pod \"data-science-collector-collector-0\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.142406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.142384    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:01:49.320099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.320076    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 11:01:49.443071 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.443049    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:01:49.444993 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:01:49.444973    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a878c7_736f_42c8_a764_e4c3fddddeb8.slice/crio-6fffee8cbfb0388891aed384bff0df5ece23d3f5508b6b546387d9f4f692cb4c WatchSource:0}: Error finding container 6fffee8cbfb0388891aed384bff0df5ece23d3f5508b6b546387d9f4f692cb4c: Status 404 returned error can't find the container with id 6fffee8cbfb0388891aed384bff0df5ece23d3f5508b6b546387d9f4f692cb4c
Oct 30 11:01:49.447178 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.447149    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:01:49.447244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.447201    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:01:49.507998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.507976    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6906bc04-cba4-40e3-87d0-56445f5fee33-tls\" (UniqueName: \"kubernetes.io/secret/6906bc04-cba4-40e3-87d0-56445f5fee33-tls\") pod \"6906bc04-cba4-40e3-87d0-56445f5fee33\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") "
Oct 30 11:01:49.508108 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.508005    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6906bc04-cba4-40e3-87d0-56445f5fee33-web-config\" (UniqueName: \"kubernetes.io/configmap/6906bc04-cba4-40e3-87d0-56445f5fee33-web-config\") pod \"6906bc04-cba4-40e3-87d0-56445f5fee33\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") "
Oct 30 11:01:49.508108 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.508054    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6906bc04-cba4-40e3-87d0-56445f5fee33-data\" (UniqueName: \"kubernetes.io/empty-dir/6906bc04-cba4-40e3-87d0-56445f5fee33-data\") pod \"6906bc04-cba4-40e3-87d0-56445f5fee33\" (UID: \"6906bc04-cba4-40e3-87d0-56445f5fee33\") "
Oct 30 11:01:49.508528 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.508503    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6906bc04-cba4-40e3-87d0-56445f5fee33-data" pod "6906bc04-cba4-40e3-87d0-56445f5fee33" (UID: "6906bc04-cba4-40e3-87d0-56445f5fee33"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:01:49.510249 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.510228    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6906bc04-cba4-40e3-87d0-56445f5fee33-tls" pod "6906bc04-cba4-40e3-87d0-56445f5fee33" (UID: "6906bc04-cba4-40e3-87d0-56445f5fee33"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:01:49.513292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.513270    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6906bc04-cba4-40e3-87d0-56445f5fee33-web-config" pod "6906bc04-cba4-40e3-87d0-56445f5fee33" (UID: "6906bc04-cba4-40e3-87d0-56445f5fee33"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:01:49.609164 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.609090    4849 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/6906bc04-cba4-40e3-87d0-56445f5fee33-tls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:49.609164 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.609122    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/configmap/6906bc04-cba4-40e3-87d0-56445f5fee33-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:49.609164 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.609136    4849 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6906bc04-cba4-40e3-87d0-56445f5fee33-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:01:49.737953 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.737918    4849 generic.go:356] "Generic (PLEG): container finished" podID="6906bc04-cba4-40e3-87d0-56445f5fee33" containerID="ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953" exitCode=0
Oct 30 11:01:49.738124 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.737976    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"
Oct 30 11:01:49.738124 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.737989    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter" event={"ID":"6906bc04-cba4-40e3-87d0-56445f5fee33","Type":"ContainerDied","Data":"ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953"}
Oct 30 11:01:49.738124 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.738082    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter" event={"ID":"6906bc04-cba4-40e3-87d0-56445f5fee33","Type":"ContainerDied","Data":"ae98c57e653aefd1e32b476119ccdbec22cfb64b8a2784a3dba7e2aa80358211"}
Oct 30 11:01:49.738124 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.738106    4849 scope.go:122] "RemoveContainer" containerID="ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953"
Oct 30 11:01:49.739753 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.739725    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e8a878c7-736f-42c8-a764-e4c3fddddeb8","Type":"ContainerStarted","Data":"13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a"}
Oct 30 11:01:49.739834 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.739772    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e8a878c7-736f-42c8-a764-e4c3fddddeb8","Type":"ContainerStarted","Data":"6fffee8cbfb0388891aed384bff0df5ece23d3f5508b6b546387d9f4f692cb4c"}
Oct 30 11:01:49.747308 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.747291    4849 scope.go:122] "RemoveContainer" containerID="ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953"
Oct 30 11:01:49.747558 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:01:49.747539    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953\": container with ID starting with ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953 not found: ID does not exist" containerID="ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953"
Oct 30 11:01:49.747615 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.747566    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953"} err="failed to get container status \"ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953\": rpc error: code = NotFound desc = could not find container \"ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953\": container with ID starting with ddb0bf9022d77defdaf9815d045e47e9c582ac0c7557f347dd8fcceb392b3953 not found: ID does not exist"
Oct 30 11:01:49.805056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.804991    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.804973925 podStartE2EDuration="1.804973925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:01:49.803511525 +0000 UTC m=+1007.183960965" watchObservedRunningTime="2026-10-30 11:01:49.804973925 +0000 UTC m=+1007.185423365"
Oct 30 11:01:49.847653 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.847627    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"]
Oct 30 11:01:49.851810 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:49.851785    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["tests-monitoring-dcgm-https-18e34997bcb25330/dcgm-https-exporter"]
Oct 30 11:01:50.755746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:50.755713    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6906bc04-cba4-40e3-87d0-56445f5fee33" path="/var/lib/kubelet/pods/6906bc04-cba4-40e3-87d0-56445f5fee33/volumes"
Oct 30 11:01:50.756219 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:01:50.756154    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c759452-49a0-4a19-a9e6-8eae97e084ca" path="/var/lib/kubelet/pods/6c759452-49a0-4a19-a9e6-8eae97e084ca/volumes"
Oct 30 11:02:02.719928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.719896    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"]
Oct 30 11:02:02.720328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.720259    4849 cpu_manager.go:414] "containerMap: removing container" podUID="6906bc04-cba4-40e3-87d0-56445f5fee33" containerName="dcgm-https-exporter"
Oct 30 11:02:02.720328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.720270    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6906bc04-cba4-40e3-87d0-56445f5fee33" containerName="dcgm-https-exporter"
Oct 30 11:02:02.720393 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.720352    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="6906bc04-cba4-40e3-87d0-56445f5fee33" containerName="dcgm-https-exporter"
Oct 30 11:02:02.724887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.724864    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:02.744798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.744774    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"]
Oct 30 11:02:02.755087 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.755067    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"tests-monitoring-tenant-metrics\"/\"default-dockercfg-s9b7r\""
Oct 30 11:02:02.755211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.755105    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"tests-monitoring-tenant-metrics\"/\"openshift-service-ca.crt\""
Oct 30 11:02:02.755211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.755145    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"tests-monitoring-tenant-metrics\"/\"kube-root-ca.crt\""
Oct 30 11:02:02.906564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.906534    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/825df71b-4a62-49fe-ba16-5b032c9c08c9-data\") pod \"tenant-metrics-exporter-5cf544fb97-m7x8z\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:02.906701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:02.906596    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jtq5\" (UniqueName: \"kubernetes.io/projected/825df71b-4a62-49fe-ba16-5b032c9c08c9-kube-api-access-4jtq5\") pod \"tenant-metrics-exporter-5cf544fb97-m7x8z\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:03.007375 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.007318    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/825df71b-4a62-49fe-ba16-5b032c9c08c9-data\") pod \"tenant-metrics-exporter-5cf544fb97-m7x8z\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:03.007375 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.007354    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jtq5\" (UniqueName: \"kubernetes.io/projected/825df71b-4a62-49fe-ba16-5b032c9c08c9-kube-api-access-4jtq5\") pod \"tenant-metrics-exporter-5cf544fb97-m7x8z\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:03.007658 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.007641    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/825df71b-4a62-49fe-ba16-5b032c9c08c9-data\") pod \"tenant-metrics-exporter-5cf544fb97-m7x8z\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:03.031486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.031464    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jtq5\" (UniqueName: \"kubernetes.io/projected/825df71b-4a62-49fe-ba16-5b032c9c08c9-kube-api-access-4jtq5\") pod \"tenant-metrics-exporter-5cf544fb97-m7x8z\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") " pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:03.037171 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.037153    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:02:03.335137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.335113    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"]
Oct 30 11:02:03.337266 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:02:03.337239    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825df71b_4a62_49fe_ba16_5b032c9c08c9.slice/crio-7e0ed47750b5a3cc9d87ffb70bfc5e3736f4264d325b1e8249c362d8cad9af64 WatchSource:0}: Error finding container 7e0ed47750b5a3cc9d87ffb70bfc5e3736f4264d325b1e8249c362d8cad9af64: Status 404 returned error can't find the container with id 7e0ed47750b5a3cc9d87ffb70bfc5e3736f4264d325b1e8249c362d8cad9af64
Oct 30 11:02:03.811427 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.811397    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z" event={"ID":"825df71b-4a62-49fe-ba16-5b032c9c08c9","Type":"ContainerStarted","Data":"009016a9d87d65db04b2a476fff74c32543ceb4b476bd3b3d9a2ec1df9259eec"}
Oct 30 11:02:03.811788 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.811445    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z" event={"ID":"825df71b-4a62-49fe-ba16-5b032c9c08c9","Type":"ContainerStarted","Data":"7e0ed47750b5a3cc9d87ffb70bfc5e3736f4264d325b1e8249c362d8cad9af64"}
Oct 30 11:02:03.856285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:03.856236    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z" podStartSLOduration=1.8562239759999999 podStartE2EDuration="1.856223976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:02:03.854370704 +0000 UTC m=+1021.234820144" watchObservedRunningTime="2026-10-30 11:02:03.856223976 +0000 UTC m=+1021.236673416"
Oct 30 11:02:04.843612 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:04.843567    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:02:04.844125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:04.843811    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="e8a878c7-736f-42c8-a764-e4c3fddddeb8" containerName="otc-container" containerID="cri-o://13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a" gracePeriod=30
Oct 30 11:02:07.189057 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.189009    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.241204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.241173    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8a878c7-736f-42c8-a764-e4c3fddddeb8-kube-api-access-zgs9v\" (UniqueName: \"kubernetes.io/projected/e8a878c7-736f-42c8-a764-e4c3fddddeb8-kube-api-access-zgs9v\") pod \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") "
Oct 30 11:02:07.241330 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.241220    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-tls-certs\") pod \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") "
Oct 30 11:02:07.241330 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.241267    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-monitoring-tls-certs\") pod \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") "
Oct 30 11:02:07.241898 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.241498    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-data-science-collector-ta-client-cert\") pod \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") "
Oct 30 11:02:07.241898 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.241584    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8a878c7-736f-42c8-a764-e4c3fddddeb8-otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8a878c7-736f-42c8-a764-e4c3fddddeb8-otc-internal\") pod \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\" (UID: \"e8a878c7-736f-42c8-a764-e4c3fddddeb8\") "
Oct 30 11:02:07.242165 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.242135    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a878c7-736f-42c8-a764-e4c3fddddeb8-otc-internal" pod "e8a878c7-736f-42c8-a764-e4c3fddddeb8" (UID: "e8a878c7-736f-42c8-a764-e4c3fddddeb8"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:02:07.244155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.244115    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-tls-certs" pod "e8a878c7-736f-42c8-a764-e4c3fddddeb8" (UID: "e8a878c7-736f-42c8-a764-e4c3fddddeb8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:02:07.244254 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.244198    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-monitoring-tls-certs" pod "e8a878c7-736f-42c8-a764-e4c3fddddeb8" (UID: "e8a878c7-736f-42c8-a764-e4c3fddddeb8"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:02:07.244341 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.244315    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-data-science-collector-ta-client-cert" pod "e8a878c7-736f-42c8-a764-e4c3fddddeb8" (UID: "e8a878c7-736f-42c8-a764-e4c3fddddeb8"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:02:07.246418 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.246397    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a878c7-736f-42c8-a764-e4c3fddddeb8-kube-api-access-zgs9v" pod "e8a878c7-736f-42c8-a764-e4c3fddddeb8" (UID: "e8a878c7-736f-42c8-a764-e4c3fddddeb8"). InnerVolumeSpecName "kube-api-access-zgs9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:02:07.342687 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.342661    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgs9v\" (UniqueName: \"kubernetes.io/projected/e8a878c7-736f-42c8-a764-e4c3fddddeb8-kube-api-access-zgs9v\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:02:07.342687 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.342683    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:02:07.342687 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.342692    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:02:07.342844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.342701    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8a878c7-736f-42c8-a764-e4c3fddddeb8-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:02:07.342844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.342711    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8a878c7-736f-42c8-a764-e4c3fddddeb8-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:02:07.837340 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.837306    4849 generic.go:356] "Generic (PLEG): container finished" podID="e8a878c7-736f-42c8-a764-e4c3fddddeb8" containerID="13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a" exitCode=0
Oct 30 11:02:07.837516 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.837359    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e8a878c7-736f-42c8-a764-e4c3fddddeb8","Type":"ContainerDied","Data":"13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a"}
Oct 30 11:02:07.837516 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.837391    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.837516 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.837410    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e8a878c7-736f-42c8-a764-e4c3fddddeb8","Type":"ContainerDied","Data":"6fffee8cbfb0388891aed384bff0df5ece23d3f5508b6b546387d9f4f692cb4c"}
Oct 30 11:02:07.837516 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.837440    4849 scope.go:122] "RemoveContainer" containerID="13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a"
Oct 30 11:02:07.849522 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.849493    4849 scope.go:122] "RemoveContainer" containerID="13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a"
Oct 30 11:02:07.849827 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:02:07.849806    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a\": container with ID starting with 13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a not found: ID does not exist" containerID="13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a"
Oct 30 11:02:07.849903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.849836    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a"} err="failed to get container status \"13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a\": rpc error: code = NotFound desc = could not find container \"13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a\": container with ID starting with 13662b07e1e7281805bb94f7b458a5e8b8a60e24fe76f06d8b2d6beb435b6f0a not found: ID does not exist"
Oct 30 11:02:07.892060 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.892035    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:02:07.905520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.905498    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:02:07.931607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.931584    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:02:07.931948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.931934    4849 cpu_manager.go:414] "containerMap: removing container" podUID="e8a878c7-736f-42c8-a764-e4c3fddddeb8" containerName="otc-container"
Oct 30 11:02:07.931989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.931950    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8a878c7-736f-42c8-a764-e4c3fddddeb8" containerName="otc-container"
Oct 30 11:02:07.932065 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.932046    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8a878c7-736f-42c8-a764-e4c3fddddeb8" containerName="otc-container"
Oct 30 11:02:07.936441 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.936427    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.946692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.946672    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.946779 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.946708    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.946779 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.946726    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqlm7\" (UniqueName: \"kubernetes.io/projected/6ad24c84-443a-4144-93ff-b541be89d60f-kube-api-access-cqlm7\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.946909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.946790    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6ad24c84-443a-4144-93ff-b541be89d60f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.946909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.946861    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:07.956671 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.956647    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:02:07.957218 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.957184    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:02:07.957218 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.957200    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 11:02:07.957218 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.957194    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:02:07.957431 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.957244    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:02:07.957431 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:07.957297    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:02:08.047641 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.047614    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6ad24c84-443a-4144-93ff-b541be89d60f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.047641 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.047643    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.047796 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.047774    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.047849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.047827    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.047901 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.047847    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqlm7\" (UniqueName: \"kubernetes.io/projected/6ad24c84-443a-4144-93ff-b541be89d60f-kube-api-access-cqlm7\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.048292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.048268    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6ad24c84-443a-4144-93ff-b541be89d60f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.050348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.050326    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.050434 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.050375    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.050474 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.050436    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.071758 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.071733    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqlm7\" (UniqueName: \"kubernetes.io/projected/6ad24c84-443a-4144-93ff-b541be89d60f-kube-api-access-cqlm7\") pod \"data-science-collector-collector-0\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.248925 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.248876    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:02:08.548101 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.548076    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:02:08.550183 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:02:08.550157    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad24c84_443a_4144_93ff_b541be89d60f.slice/crio-90eedcc29cb54a0250df653859918ab0339da7fefcbddb09f25de7603606d400 WatchSource:0}: Error finding container 90eedcc29cb54a0250df653859918ab0339da7fefcbddb09f25de7603606d400: Status 404 returned error can't find the container with id 90eedcc29cb54a0250df653859918ab0339da7fefcbddb09f25de7603606d400
Oct 30 11:02:08.552334 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.552304    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:02:08.552394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.552367    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:02:08.753734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.753700    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8a878c7-736f-42c8-a764-e4c3fddddeb8" path="/var/lib/kubelet/pods/e8a878c7-736f-42c8-a764-e4c3fddddeb8/volumes"
Oct 30 11:02:08.845213 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.845184    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6ad24c84-443a-4144-93ff-b541be89d60f","Type":"ContainerStarted","Data":"37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980"}
Oct 30 11:02:08.845367 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.845244    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6ad24c84-443a-4144-93ff-b541be89d60f","Type":"ContainerStarted","Data":"90eedcc29cb54a0250df653859918ab0339da7fefcbddb09f25de7603606d400"}
Oct 30 11:02:08.891390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:02:08.891334    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.891316045 podStartE2EDuration="1.891316045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:02:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:02:08.888912644 +0000 UTC m=+1026.269362085" watchObservedRunningTime="2026-10-30 11:02:08.891316045 +0000 UTC m=+1026.271765486"
Oct 30 11:02:09.432161 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:02:09.432129    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:02:39.430555 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:02:39.430484    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:03:09.431998 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:03:09.431957    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:03:39.431702 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:03:39.431666    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:04:09.433535 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:04:09.433493    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:04:39.431235 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:04:39.431191    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:05:02.882168 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:02.882141    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:05:02.884778 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:02.884017    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:05:09.431711 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:05:09.431660    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:05:39.432709 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:05:39.432623    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:05:45.868864 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:45.868820    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"]
Oct 30 11:05:45.869517 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:45.869085    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z" podUID="825df71b-4a62-49fe-ba16-5b032c9c08c9" containerName="tenant-metrics-exporter" containerID="cri-o://009016a9d87d65db04b2a476fff74c32543ceb4b476bd3b3d9a2ec1df9259eec" gracePeriod=30
Oct 30 11:05:45.986675 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:45.986651    4849 generic.go:356] "Generic (PLEG): container finished" podID="825df71b-4a62-49fe-ba16-5b032c9c08c9" containerID="009016a9d87d65db04b2a476fff74c32543ceb4b476bd3b3d9a2ec1df9259eec" exitCode=0
Oct 30 11:05:45.986777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:45.986719    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z" event={"ID":"825df71b-4a62-49fe-ba16-5b032c9c08c9","Type":"ContainerDied","Data":"009016a9d87d65db04b2a476fff74c32543ceb4b476bd3b3d9a2ec1df9259eec"}
Oct 30 11:05:46.123689 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.123637    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:05:46.211445 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.211420    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/825df71b-4a62-49fe-ba16-5b032c9c08c9-kube-api-access-4jtq5\" (UniqueName: \"kubernetes.io/projected/825df71b-4a62-49fe-ba16-5b032c9c08c9-kube-api-access-4jtq5\") pod \"825df71b-4a62-49fe-ba16-5b032c9c08c9\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") "
Oct 30 11:05:46.211566 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.211510    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/825df71b-4a62-49fe-ba16-5b032c9c08c9-data\" (UniqueName: \"kubernetes.io/empty-dir/825df71b-4a62-49fe-ba16-5b032c9c08c9-data\") pod \"825df71b-4a62-49fe-ba16-5b032c9c08c9\" (UID: \"825df71b-4a62-49fe-ba16-5b032c9c08c9\") "
Oct 30 11:05:46.211877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.211854    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825df71b-4a62-49fe-ba16-5b032c9c08c9-data" pod "825df71b-4a62-49fe-ba16-5b032c9c08c9" (UID: "825df71b-4a62-49fe-ba16-5b032c9c08c9"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:05:46.213539 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.213518    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/825df71b-4a62-49fe-ba16-5b032c9c08c9-kube-api-access-4jtq5" pod "825df71b-4a62-49fe-ba16-5b032c9c08c9" (UID: "825df71b-4a62-49fe-ba16-5b032c9c08c9"). InnerVolumeSpecName "kube-api-access-4jtq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:05:46.312705 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.312681    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jtq5\" (UniqueName: \"kubernetes.io/projected/825df71b-4a62-49fe-ba16-5b032c9c08c9-kube-api-access-4jtq5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:05:46.312811 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.312710    4849 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/825df71b-4a62-49fe-ba16-5b032c9c08c9-data\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:05:46.993748 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.993714    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z" event={"ID":"825df71b-4a62-49fe-ba16-5b032c9c08c9","Type":"ContainerDied","Data":"7e0ed47750b5a3cc9d87ffb70bfc5e3736f4264d325b1e8249c362d8cad9af64"}
Oct 30 11:05:46.993748 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.993745    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"
Oct 30 11:05:46.994301 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:46.993776    4849 scope.go:122] "RemoveContainer" containerID="009016a9d87d65db04b2a476fff74c32543ceb4b476bd3b3d9a2ec1df9259eec"
Oct 30 11:05:47.048253 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:47.048196    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"]
Oct 30 11:05:47.054104 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:47.054083    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["tests-monitoring-tenant-metrics/tenant-metrics-exporter-5cf544fb97-m7x8z"]
Oct 30 11:05:48.753741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:05:48.753702    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="825df71b-4a62-49fe-ba16-5b032c9c08c9" path="/var/lib/kubelet/pods/825df71b-4a62-49fe-ba16-5b032c9c08c9/volumes"
Oct 30 11:06:05.528993 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.528961    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-9bdc7-5svrl"]
Oct 30 11:06:05.529423 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.529381    4849 cpu_manager.go:414] "containerMap: removing container" podUID="825df71b-4a62-49fe-ba16-5b032c9c08c9" containerName="tenant-metrics-exporter"
Oct 30 11:06:05.529423 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.529396    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="825df71b-4a62-49fe-ba16-5b032c9c08c9" containerName="tenant-metrics-exporter"
Oct 30 11:06:05.529499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.529476    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="825df71b-4a62-49fe-ba16-5b032c9c08c9" containerName="tenant-metrics-exporter"
Oct 30 11:06:05.533786 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.533765    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.551450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.551423    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-9bdc7-5svrl"]
Oct 30 11:06:05.563265 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.563246    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wj9xd\""
Oct 30 11:06:05.650541 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.650507    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-trusted-ca-bundle\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.650675 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.650560    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-archives-path\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.650746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.650704    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-service-ca-bundle\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.650829 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.650808    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbrtm\" (UniqueName: \"kubernetes.io/projected/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-kube-api-access-qbrtm\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.752130 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.752102    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-service-ca-bundle\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.752243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.752152    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbrtm\" (UniqueName: \"kubernetes.io/projected/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-kube-api-access-qbrtm\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.752243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.752177    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-trusted-ca-bundle\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.752243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.752203    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-archives-path\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.752618 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.752602    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-archives-path\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.752837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.752817    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-service-ca-bundle\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.752920 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.752901    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-trusted-ca-bundle\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.776977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.776954    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbrtm\" (UniqueName: \"kubernetes.io/projected/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-kube-api-access-qbrtm\") pod \"periodic-gathering-9bdc7-5svrl\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") " pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:05.846193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:05.846171    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:06:06.144259 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:06.144234    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-9bdc7-5svrl"]
Oct 30 11:06:06.146427 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:06:06.146397    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b1ffda_63a4_455d_a5ef_f6243dcf2480.slice/crio-c32a354f5b8b9720df47bddae142e01ceb22d901bb62e59a6986c3379ba26fc7 WatchSource:0}: Error finding container c32a354f5b8b9720df47bddae142e01ceb22d901bb62e59a6986c3379ba26fc7: Status 404 returned error can't find the container with id c32a354f5b8b9720df47bddae142e01ceb22d901bb62e59a6986c3379ba26fc7
Oct 30 11:06:07.089423 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:07.089343    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:06:07.089866 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:07.089619    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="6ad24c84-443a-4144-93ff-b541be89d60f" containerName="otc-container" containerID="cri-o://37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980" gracePeriod=30
Oct 30 11:06:07.099777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:07.099744    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9bdc7-5svrl" event={"ID":"b5b1ffda-63a4-455d-a5ef-f6243dcf2480","Type":"ContainerStarted","Data":"03769ffc1f1376975efa0596fb549c7a9028a62e3d6b90ae3b7bb379483ae4d3"}
Oct 30 11:06:07.099917 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:07.099807    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9bdc7-5svrl" event={"ID":"b5b1ffda-63a4-455d-a5ef-f6243dcf2480","Type":"ContainerStarted","Data":"c32a354f5b8b9720df47bddae142e01ceb22d901bb62e59a6986c3379ba26fc7"}
Oct 30 11:06:07.175722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:07.175679    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-9bdc7-5svrl" podStartSLOduration=2.175665565 podStartE2EDuration="2.175665565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:06:07.175231117 +0000 UTC m=+1264.555680784" watchObservedRunningTime="2026-10-30 11:06:07.175665565 +0000 UTC m=+1264.556115040"
Oct 30 11:06:09.432347 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:06:09.432316    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:06:09.440563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.440543    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:09.584371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.584340    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-data-science-collector-ta-client-cert\") pod \"6ad24c84-443a-4144-93ff-b541be89d60f\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") "
Oct 30 11:06:09.584521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.584381    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ad24c84-443a-4144-93ff-b541be89d60f-otc-internal\" (UniqueName: \"kubernetes.io/configmap/6ad24c84-443a-4144-93ff-b541be89d60f-otc-internal\") pod \"6ad24c84-443a-4144-93ff-b541be89d60f\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") "
Oct 30 11:06:09.584521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.584405    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-monitoring-tls-certs\") pod \"6ad24c84-443a-4144-93ff-b541be89d60f\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") "
Oct 30 11:06:09.584652 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.584532    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ad24c84-443a-4144-93ff-b541be89d60f-kube-api-access-cqlm7\" (UniqueName: \"kubernetes.io/projected/6ad24c84-443a-4144-93ff-b541be89d60f-kube-api-access-cqlm7\") pod \"6ad24c84-443a-4144-93ff-b541be89d60f\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") "
Oct 30 11:06:09.584652 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.584567    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-tls-certs\") pod \"6ad24c84-443a-4144-93ff-b541be89d60f\" (UID: \"6ad24c84-443a-4144-93ff-b541be89d60f\") "
Oct 30 11:06:09.584829 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.584792    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad24c84-443a-4144-93ff-b541be89d60f-otc-internal" pod "6ad24c84-443a-4144-93ff-b541be89d60f" (UID: "6ad24c84-443a-4144-93ff-b541be89d60f"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:06:09.586734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.586707    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad24c84-443a-4144-93ff-b541be89d60f-kube-api-access-cqlm7" pod "6ad24c84-443a-4144-93ff-b541be89d60f" (UID: "6ad24c84-443a-4144-93ff-b541be89d60f"). InnerVolumeSpecName "kube-api-access-cqlm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:06:09.586856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.586715    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-tls-certs" pod "6ad24c84-443a-4144-93ff-b541be89d60f" (UID: "6ad24c84-443a-4144-93ff-b541be89d60f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:06:09.586856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.586712    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-monitoring-tls-certs" pod "6ad24c84-443a-4144-93ff-b541be89d60f" (UID: "6ad24c84-443a-4144-93ff-b541be89d60f"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:06:09.586856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.586771    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-data-science-collector-ta-client-cert" pod "6ad24c84-443a-4144-93ff-b541be89d60f" (UID: "6ad24c84-443a-4144-93ff-b541be89d60f"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:06:09.685846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.685821    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/6ad24c84-443a-4144-93ff-b541be89d60f-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:06:09.685846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.685844    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:06:09.685968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.685853    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqlm7\" (UniqueName: \"kubernetes.io/projected/6ad24c84-443a-4144-93ff-b541be89d60f-kube-api-access-cqlm7\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:06:09.685968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.685862    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:06:09.685968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:09.685871    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/6ad24c84-443a-4144-93ff-b541be89d60f-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:06:10.116846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.116819    4849 generic.go:356] "Generic (PLEG): container finished" podID="6ad24c84-443a-4144-93ff-b541be89d60f" containerID="37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980" exitCode=0
Oct 30 11:06:10.116974 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.116862    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.116974 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.116899    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6ad24c84-443a-4144-93ff-b541be89d60f","Type":"ContainerDied","Data":"37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980"}
Oct 30 11:06:10.116974 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.116951    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"6ad24c84-443a-4144-93ff-b541be89d60f","Type":"ContainerDied","Data":"90eedcc29cb54a0250df653859918ab0339da7fefcbddb09f25de7603606d400"}
Oct 30 11:06:10.116974 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.116972    4849 scope.go:122] "RemoveContainer" containerID="37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980"
Oct 30 11:06:10.126793 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.126772    4849 scope.go:122] "RemoveContainer" containerID="37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980"
Oct 30 11:06:10.127094 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:06:10.127076    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980\": container with ID starting with 37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980 not found: ID does not exist" containerID="37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980"
Oct 30 11:06:10.127148 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.127103    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980"} err="failed to get container status \"37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980\": rpc error: code = NotFound desc = could not find container \"37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980\": container with ID starting with 37837f5583190be8608e9b45921e6bef185263318d1287eb49c067b350db0980 not found: ID does not exist"
Oct 30 11:06:10.171847 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.171824    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:06:10.185364 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.185337    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:06:10.214089 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.214071    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:06:10.214422 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.214411    4849 cpu_manager.go:414] "containerMap: removing container" podUID="6ad24c84-443a-4144-93ff-b541be89d60f" containerName="otc-container"
Oct 30 11:06:10.214465 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.214424    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ad24c84-443a-4144-93ff-b541be89d60f" containerName="otc-container"
Oct 30 11:06:10.214500 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.214489    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ad24c84-443a-4144-93ff-b541be89d60f" containerName="otc-container"
Oct 30 11:06:10.218754 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.218735    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.237392 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.237374    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:06:10.237583 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.237564    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vrjb\""
Oct 30 11:06:10.237656 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.237644    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:06:10.237707 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.237696    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:06:10.237877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.237850    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:06:10.239327 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.239305    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:06:10.390688 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.390627    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.390688 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.390657    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.390852 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.390702    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.390852 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.390762    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqskd\" (UniqueName: \"kubernetes.io/projected/98cd68cb-5d87-42af-8d7d-4f842be844b9-kube-api-access-hqskd\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.390964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.390877    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/98cd68cb-5d87-42af-8d7d-4f842be844b9-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.491765 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.491738    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/98cd68cb-5d87-42af-8d7d-4f842be844b9-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.492186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.491775    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.492186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.491796    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.492186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.491851    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.492186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.491886    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqskd\" (UniqueName: \"kubernetes.io/projected/98cd68cb-5d87-42af-8d7d-4f842be844b9-kube-api-access-hqskd\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.492559 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.492537    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/98cd68cb-5d87-42af-8d7d-4f842be844b9-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.494391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.494371    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.494489 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.494469    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.494550 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.494522    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.516759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.516731    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqskd\" (UniqueName: \"kubernetes.io/projected/98cd68cb-5d87-42af-8d7d-4f842be844b9-kube-api-access-hqskd\") pod \"data-science-collector-collector-0\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.530415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.530396    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:06:10.753681 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.753613    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ad24c84-443a-4144-93ff-b541be89d60f" path="/var/lib/kubelet/pods/6ad24c84-443a-4144-93ff-b541be89d60f/volumes"
Oct 30 11:06:10.829120 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.829096    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:06:10.830937 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:06:10.830914    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98cd68cb_5d87_42af_8d7d_4f842be844b9.slice/crio-afd4fa026c373e643a7e48fd538389c9fd391275ee0aec703b17a47028be19f7 WatchSource:0}: Error finding container afd4fa026c373e643a7e48fd538389c9fd391275ee0aec703b17a47028be19f7: Status 404 returned error can't find the container with id afd4fa026c373e643a7e48fd538389c9fd391275ee0aec703b17a47028be19f7
Oct 30 11:06:10.832938 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.832904    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:06:10.833051 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:10.832973    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:06:11.123714 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:11.123678    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"98cd68cb-5d87-42af-8d7d-4f842be844b9","Type":"ContainerStarted","Data":"fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38"}
Oct 30 11:06:11.123879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:11.123735    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"98cd68cb-5d87-42af-8d7d-4f842be844b9","Type":"ContainerStarted","Data":"afd4fa026c373e643a7e48fd538389c9fd391275ee0aec703b17a47028be19f7"}
Oct 30 11:06:11.169268 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:11.169207    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.169193977 podStartE2EDuration="1.169193977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:06:11.167017921 +0000 UTC m=+1268.547467360" watchObservedRunningTime="2026-10-30 11:06:11.169193977 +0000 UTC m=+1268.549643416"
Oct 30 11:06:23.846210 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:23.846170    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"]
Oct 30 11:06:23.849784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:23.849764    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"
Oct 30 11:06:23.867765 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:23.867732    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"]
Oct 30 11:06:23.878734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:23.878715    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-kh8sh\""
Oct 30 11:06:23.882826 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:23.882801    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"
Oct 30 11:06:24.181418 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:24.181392    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"]
Oct 30 11:06:24.183255 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:06:24.183225    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod282f747d_fc70_463d_b2a9_683add07f4c8.slice/crio-70ccb382c13682ba8422748d7f9ca9c6f3919d5ad6beb85a36c688e9ed36adde WatchSource:0}: Error finding container 70ccb382c13682ba8422748d7f9ca9c6f3919d5ad6beb85a36c688e9ed36adde: Status 404 returned error can't find the container with id 70ccb382c13682ba8422748d7f9ca9c6f3919d5ad6beb85a36c688e9ed36adde
Oct 30 11:06:24.190401 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:24.190374    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd" event={"ID":"282f747d-fc70-463d-b2a9-683add07f4c8","Type":"ContainerStarted","Data":"70ccb382c13682ba8422748d7f9ca9c6f3919d5ad6beb85a36c688e9ed36adde"}
Oct 30 11:06:25.197559 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:25.197522    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd" event={"ID":"282f747d-fc70-463d-b2a9-683add07f4c8","Type":"ContainerStarted","Data":"309dcddef4f778a9cbb44be3bda4c8a6cb12e516f831482164581822cb035770"}
Oct 30 11:06:25.237463 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:25.237405    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd" podStartSLOduration=2.2373894229999998 podStartE2EDuration="2.237389423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:06:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:06:25.237383856 +0000 UTC m=+1282.617833298" watchObservedRunningTime="2026-10-30 11:06:25.237389423 +0000 UTC m=+1282.617838864"
Oct 30 11:06:35.263859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:35.263822    4849 generic.go:356] "Generic (PLEG): container finished" podID="282f747d-fc70-463d-b2a9-683add07f4c8" containerID="309dcddef4f778a9cbb44be3bda4c8a6cb12e516f831482164581822cb035770" exitCode=0
Oct 30 11:06:35.264290 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:35.263889    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd" event={"ID":"282f747d-fc70-463d-b2a9-683add07f4c8","Type":"ContainerDied","Data":"309dcddef4f778a9cbb44be3bda4c8a6cb12e516f831482164581822cb035770"}
Oct 30 11:06:36.403093 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:36.403068    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"
Oct 30 11:06:37.275531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:37.275507    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"
Oct 30 11:06:37.275732 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:37.275550    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd" event={"ID":"282f747d-fc70-463d-b2a9-683add07f4c8","Type":"ContainerDied","Data":"70ccb382c13682ba8422748d7f9ca9c6f3919d5ad6beb85a36c688e9ed36adde"}
Oct 30 11:06:37.275732 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:06:37.275595    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70ccb382c13682ba8422748d7f9ca9c6f3919d5ad6beb85a36c688e9ed36adde"
Oct 30 11:06:39.430368 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:06:39.430333    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:07:09.433089 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:07:09.433017    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:07:39.429270 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:07:39.429237    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:07:39.610349 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:39.610318    4849 generic.go:356] "Generic (PLEG): container finished" podID="b5b1ffda-63a4-455d-a5ef-f6243dcf2480" containerID="03769ffc1f1376975efa0596fb549c7a9028a62e3d6b90ae3b7bb379483ae4d3" exitCode=0
Oct 30 11:07:39.610490 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:39.610389    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9bdc7-5svrl" event={"ID":"b5b1ffda-63a4-455d-a5ef-f6243dcf2480","Type":"ContainerDied","Data":"03769ffc1f1376975efa0596fb549c7a9028a62e3d6b90ae3b7bb379483ae4d3"}
Oct 30 11:07:40.771048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.771006    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:07:40.880231 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.880171    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-archives-path\") pod \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") "
Oct 30 11:07:40.880231 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.880204    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-kube-api-access-qbrtm\" (UniqueName: \"kubernetes.io/projected/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-kube-api-access-qbrtm\") pod \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") "
Oct 30 11:07:40.880418 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.880256    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-trusted-ca-bundle\") pod \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") "
Oct 30 11:07:40.880418 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.880277    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-service-ca-bundle\") pod \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\" (UID: \"b5b1ffda-63a4-455d-a5ef-f6243dcf2480\") "
Oct 30 11:07:40.880541 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.880512    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-archives-path" pod "b5b1ffda-63a4-455d-a5ef-f6243dcf2480" (UID: "b5b1ffda-63a4-455d-a5ef-f6243dcf2480"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:07:40.880683 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.880662    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-trusted-ca-bundle" pod "b5b1ffda-63a4-455d-a5ef-f6243dcf2480" (UID: "b5b1ffda-63a4-455d-a5ef-f6243dcf2480"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:07:40.880744 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.880687    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-service-ca-bundle" pod "b5b1ffda-63a4-455d-a5ef-f6243dcf2480" (UID: "b5b1ffda-63a4-455d-a5ef-f6243dcf2480"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:07:40.882380 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.882361    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-kube-api-access-qbrtm" pod "b5b1ffda-63a4-455d-a5ef-f6243dcf2480" (UID: "b5b1ffda-63a4-455d-a5ef-f6243dcf2480"). InnerVolumeSpecName "kube-api-access-qbrtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:07:40.981675 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.981643    4849 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-service-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:07:40.981675 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.981677    4849 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-archives-path\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:07:40.981804 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.981687    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbrtm\" (UniqueName: \"kubernetes.io/projected/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-kube-api-access-qbrtm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:07:40.981804 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:40.981696    4849 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5b1ffda-63a4-455d-a5ef-f6243dcf2480-trusted-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:07:41.623406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:41.623373    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9bdc7-5svrl"
Oct 30 11:07:41.623406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:41.623388    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9bdc7-5svrl" event={"ID":"b5b1ffda-63a4-455d-a5ef-f6243dcf2480","Type":"ContainerDied","Data":"c32a354f5b8b9720df47bddae142e01ceb22d901bb62e59a6986c3379ba26fc7"}
Oct 30 11:07:41.623597 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:07:41.623436    4849 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c32a354f5b8b9720df47bddae142e01ceb22d901bb62e59a6986c3379ba26fc7"
Oct 30 11:08:09.432351 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:08:09.432307    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:08:39.431352 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:08:39.431270    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:09:09.432808 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:09:09.432767    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:09:39.431864 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:09:39.431828    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:10:02.915800 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:10:02.915693    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:10:02.919145 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:10:02.919122    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:10:09.434159 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:10:09.434118    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:10:39.430637 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:10:39.430602    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:10:58.054402 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:10:58.054370    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"]
Oct 30 11:10:58.062930 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:10:58.062903    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-untrusted-denied-18e349f2ca604efd"]
Oct 30 11:10:58.753936 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:10:58.753901    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="282f747d-fc70-463d-b2a9-683add07f4c8" path="/var/lib/kubelet/pods/282f747d-fc70-463d-b2a9-683add07f4c8/volumes"
Oct 30 11:11:09.432904 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:09.432860    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:11:34.122498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.122400    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:11:34.123092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.122748    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="prometheus" containerID="cri-o://bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d" gracePeriod=600
Oct 30 11:11:34.123092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.122798    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="config-reloader" containerID="cri-o://aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6" gracePeriod=600
Oct 30 11:11:34.123092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.122791    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="thanos-sidecar" containerID="cri-o://72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836" gracePeriod=600
Oct 30 11:11:34.149237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.149207    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:11:34.149578 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.149531    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="alertmanager" containerID="cri-o://42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda" gracePeriod=120
Oct 30 11:11:34.149641 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.149582    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="config-reloader" containerID="cri-o://ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee" gracePeriod=120
Oct 30 11:11:34.666573 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.666549    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:11:34.763784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.763694    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.763784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.763734    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-secret-prometheus-web-tls-ca\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.763831    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.763865    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.763897    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.763937    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-tls-assets\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-tls-assets\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.763968    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.764046    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.764078    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-thanos-prometheus-http-client-file\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.764139    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config-out\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.764188    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-kube-api-access-gbrmv\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-kube-api-access-gbrmv\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.764223    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.764348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.764254    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\" (UID: \"ef541574-0c7d-4a58-b3fb-4cf1f820217c\") "
Oct 30 11:11:34.766569 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.766537    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-1" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:11:34.766973 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.766946    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-0" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:11:34.767123 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.767079    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-2" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:11:34.770290 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.770233    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config-out" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:11:34.770968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.770901    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:34.772691 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.772662    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-secret-prometheus-web-tls-ca" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:34.774238 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.774202    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-kube-api-access-gbrmv" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "kube-api-access-gbrmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:11:34.774238 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.774198    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:34.774238 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.774225    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-thanos-prometheus-http-client-file" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:34.774406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.774260    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-tls-assets" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:11:34.774406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.774267    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:34.777794 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.777775    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:11:34.786836 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.786811    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config" pod "ef541574-0c7d-4a58-b3fb-4cf1f820217c" (UID: "ef541574-0c7d-4a58-b3fb-4cf1f820217c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:34.817087 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.817063    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:11:34.817296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.817258    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="98cd68cb-5d87-42af-8d7d-4f842be844b9" containerName="otc-container" containerID="cri-o://fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38" gracePeriod=30
Oct 30 11:11:34.865733 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865699    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865733 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865736    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865751    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865765    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbrmv\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-kube-api-access-gbrmv\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865781    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865798    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865814    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865828    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865867    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865885    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865899    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef541574-0c7d-4a58-b3fb-4cf1f820217c-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865912    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef541574-0c7d-4a58-b3fb-4cf1f820217c-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.865928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.865926    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef541574-0c7d-4a58-b3fb-4cf1f820217c-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:34.884845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.884815    4849 generic.go:356] "Generic (PLEG): container finished" podID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerID="72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836" exitCode=0
Oct 30 11:11:34.884845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.884842    4849 generic.go:356] "Generic (PLEG): container finished" podID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerID="aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6" exitCode=0
Oct 30 11:11:34.885055 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.884850    4849 generic.go:356] "Generic (PLEG): container finished" podID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerID="bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d" exitCode=0
Oct 30 11:11:34.885055 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.884893    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerDied","Data":"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"}
Oct 30 11:11:34.885055 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.884910    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:11:34.885055 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.884993    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerDied","Data":"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"}
Oct 30 11:11:34.885055 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.885045    4849 scope.go:122] "RemoveContainer" containerID="72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"
Oct 30 11:11:34.885301 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.885067    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerDied","Data":"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"}
Oct 30 11:11:34.885301 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.885105    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"ef541574-0c7d-4a58-b3fb-4cf1f820217c","Type":"ContainerDied","Data":"849cf0ddbced9405aba93026c85f4610f2c39a113a0ea9ae0f64336d678df884"}
Oct 30 11:11:34.887410 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.887390    4849 generic.go:356] "Generic (PLEG): container finished" podID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerID="ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee" exitCode=0
Oct 30 11:11:34.887527 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.887457    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerDied","Data":"ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee"}
Oct 30 11:11:34.897999 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.897986    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:11:34.942794 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.942769    4849 scope.go:122] "RemoveContainer" containerID="aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"
Oct 30 11:11:34.960607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.960585    4849 scope.go:122] "RemoveContainer" containerID="bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"
Oct 30 11:11:34.967052 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.967016    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\""
Oct 30 11:11:34.970421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.970336    4849 scope.go:122] "RemoveContainer" containerID="2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"
Oct 30 11:11:34.980613 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.980593    4849 scope.go:122] "RemoveContainer" containerID="72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"
Oct 30 11:11:34.980938 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:34.980918    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836\": container with ID starting with 72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836 not found: ID does not exist" containerID="72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"
Oct 30 11:11:34.981060 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.980948    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"} err="failed to get container status \"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836\": rpc error: code = NotFound desc = could not find container \"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836\": container with ID starting with 72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836 not found: ID does not exist"
Oct 30 11:11:34.981060 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.980966    4849 scope.go:122] "RemoveContainer" containerID="aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"
Oct 30 11:11:34.983094 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:34.981471    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6\": container with ID starting with aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6 not found: ID does not exist" containerID="aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"
Oct 30 11:11:34.983094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.982345    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"} err="failed to get container status \"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6\": rpc error: code = NotFound desc = could not find container \"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6\": container with ID starting with aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6 not found: ID does not exist"
Oct 30 11:11:34.983094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.982369    4849 scope.go:122] "RemoveContainer" containerID="bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"
Oct 30 11:11:34.983094 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:34.982703    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d\": container with ID starting with bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d not found: ID does not exist" containerID="bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"
Oct 30 11:11:34.983094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.982736    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"} err="failed to get container status \"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d\": rpc error: code = NotFound desc = could not find container \"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d\": container with ID starting with bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d not found: ID does not exist"
Oct 30 11:11:34.983094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.982778    4849 scope.go:122] "RemoveContainer" containerID="2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"
Oct 30 11:11:34.983494 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:34.983113    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa\": container with ID starting with 2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa not found: ID does not exist" containerID="2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"
Oct 30 11:11:34.983494 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.983142    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"} err="failed to get container status \"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa\": rpc error: code = NotFound desc = could not find container \"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa\": container with ID starting with 2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa not found: ID does not exist"
Oct 30 11:11:34.983494 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.983162    4849 scope.go:122] "RemoveContainer" containerID="72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"
Oct 30 11:11:34.983607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.983518    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"} err="failed to get container status \"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836\": rpc error: code = NotFound desc = could not find container \"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836\": container with ID starting with 72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836 not found: ID does not exist"
Oct 30 11:11:34.983607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.983535    4849 scope.go:122] "RemoveContainer" containerID="aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"
Oct 30 11:11:34.983773 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.983751    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"} err="failed to get container status \"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6\": rpc error: code = NotFound desc = could not find container \"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6\": container with ID starting with aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6 not found: ID does not exist"
Oct 30 11:11:34.983773 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.983771    4849 scope.go:122] "RemoveContainer" containerID="bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"
Oct 30 11:11:34.984075 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984050    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"} err="failed to get container status \"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d\": rpc error: code = NotFound desc = could not find container \"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d\": container with ID starting with bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d not found: ID does not exist"
Oct 30 11:11:34.984141 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984077    4849 scope.go:122] "RemoveContainer" containerID="2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"
Oct 30 11:11:34.984364 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984337    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"} err="failed to get container status \"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa\": rpc error: code = NotFound desc = could not find container \"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa\": container with ID starting with 2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa not found: ID does not exist"
Oct 30 11:11:34.984364 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984358    4849 scope.go:122] "RemoveContainer" containerID="72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"
Oct 30 11:11:34.984596 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984578    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836"} err="failed to get container status \"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836\": rpc error: code = NotFound desc = could not find container \"72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836\": container with ID starting with 72891b5fa0bc9ef26229a4464c212b2e1cf76705415ff63156eac8ee2fecc836 not found: ID does not exist"
Oct 30 11:11:34.984596 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984596    4849 scope.go:122] "RemoveContainer" containerID="aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"
Oct 30 11:11:34.984848 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984832    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6"} err="failed to get container status \"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6\": rpc error: code = NotFound desc = could not find container \"aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6\": container with ID starting with aee0393961c82da0518e6208702bcce34b96092b2b692c9273c767bffd1191a6 not found: ID does not exist"
Oct 30 11:11:34.984892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.984850    4849 scope.go:122] "RemoveContainer" containerID="bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"
Oct 30 11:11:34.985117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.985091    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d"} err="failed to get container status \"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d\": rpc error: code = NotFound desc = could not find container \"bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d\": container with ID starting with bc2080d66dd52443867d5f4780ea6501bc0341e7e346e29e3dcabc500652166d not found: ID does not exist"
Oct 30 11:11:34.985117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.985117    4849 scope.go:122] "RemoveContainer" containerID="2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"
Oct 30 11:11:34.985341 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.985322    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa"} err="failed to get container status \"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa\": rpc error: code = NotFound desc = could not find container \"2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa\": container with ID starting with 2fea55ec684eb52d8995285389257231a919ba3aa440198d28407663f2a657aa not found: ID does not exist"
Oct 30 11:11:34.985687 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.985670    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:11:34.995401 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:34.995380    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:11:35.110343 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.110322    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:35.117758 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.117736    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:11:35.168509 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168479    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-out\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-out\") pod \"df960743-2f84-4069-8a1d-0c8e65af4d9b\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") "
Oct 30 11:11:35.168896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168628    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-kube-api-access-c8trw\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-kube-api-access-c8trw\") pod \"df960743-2f84-4069-8a1d-0c8e65af4d9b\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") "
Oct 30 11:11:35.168896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168665    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-volume\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-volume\") pod \"df960743-2f84-4069-8a1d-0c8e65af4d9b\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") "
Oct 30 11:11:35.168896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168698    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-data-science-collector-ta-client-cert\") pod \"98cd68cb-5d87-42af-8d7d-4f842be844b9\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") "
Oct 30 11:11:35.168896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168778    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-alertmanager-data-science-monitoringstack-db\") pod \"df960743-2f84-4069-8a1d-0c8e65af4d9b\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") "
Oct 30 11:11:35.168896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168815    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-tls-assets\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-tls-assets\") pod \"df960743-2f84-4069-8a1d-0c8e65af4d9b\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") "
Oct 30 11:11:35.168896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168878    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-monitoring-tls-certs\") pod \"98cd68cb-5d87-42af-8d7d-4f842be844b9\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") "
Oct 30 11:11:35.169252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168917    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-cluster-tls-config\") pod \"df960743-2f84-4069-8a1d-0c8e65af4d9b\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") "
Oct 30 11:11:35.169252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.168956    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98cd68cb-5d87-42af-8d7d-4f842be844b9-otc-internal\" (UniqueName: \"kubernetes.io/configmap/98cd68cb-5d87-42af-8d7d-4f842be844b9-otc-internal\") pod \"98cd68cb-5d87-42af-8d7d-4f842be844b9\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") "
Oct 30 11:11:35.169252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.169001    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-tls-certs\") pod \"98cd68cb-5d87-42af-8d7d-4f842be844b9\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") "
Oct 30 11:11:35.169252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.169060    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-web-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-web-config\") pod \"df960743-2f84-4069-8a1d-0c8e65af4d9b\" (UID: \"df960743-2f84-4069-8a1d-0c8e65af4d9b\") "
Oct 30 11:11:35.169252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.169090    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98cd68cb-5d87-42af-8d7d-4f842be844b9-kube-api-access-hqskd\" (UniqueName: \"kubernetes.io/projected/98cd68cb-5d87-42af-8d7d-4f842be844b9-kube-api-access-hqskd\") pod \"98cd68cb-5d87-42af-8d7d-4f842be844b9\" (UID: \"98cd68cb-5d87-42af-8d7d-4f842be844b9\") "
Oct 30 11:11:35.170790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.170756    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-alertmanager-data-science-monitoringstack-db" pod "df960743-2f84-4069-8a1d-0c8e65af4d9b" (UID: "df960743-2f84-4069-8a1d-0c8e65af4d9b"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:11:35.171732 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.171688    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98cd68cb-5d87-42af-8d7d-4f842be844b9-otc-internal" pod "98cd68cb-5d87-42af-8d7d-4f842be844b9" (UID: "98cd68cb-5d87-42af-8d7d-4f842be844b9"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:11:35.174186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.174147    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-tls-assets" pod "df960743-2f84-4069-8a1d-0c8e65af4d9b" (UID: "df960743-2f84-4069-8a1d-0c8e65af4d9b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:11:35.174430 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.174400    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-volume" pod "df960743-2f84-4069-8a1d-0c8e65af4d9b" (UID: "df960743-2f84-4069-8a1d-0c8e65af4d9b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:35.174726 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.174707    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-monitoring-tls-certs" pod "98cd68cb-5d87-42af-8d7d-4f842be844b9" (UID: "98cd68cb-5d87-42af-8d7d-4f842be844b9"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:35.175122 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.175102    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98cd68cb-5d87-42af-8d7d-4f842be844b9-kube-api-access-hqskd" pod "98cd68cb-5d87-42af-8d7d-4f842be844b9" (UID: "98cd68cb-5d87-42af-8d7d-4f842be844b9"). InnerVolumeSpecName "kube-api-access-hqskd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:11:35.175578 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.175554    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-kube-api-access-c8trw" pod "df960743-2f84-4069-8a1d-0c8e65af4d9b" (UID: "df960743-2f84-4069-8a1d-0c8e65af4d9b"). InnerVolumeSpecName "kube-api-access-c8trw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:11:35.176638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.176613    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-tls-certs" pod "98cd68cb-5d87-42af-8d7d-4f842be844b9" (UID: "98cd68cb-5d87-42af-8d7d-4f842be844b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:35.179863 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.179839    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-data-science-collector-ta-client-cert" pod "98cd68cb-5d87-42af-8d7d-4f842be844b9" (UID: "98cd68cb-5d87-42af-8d7d-4f842be844b9"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:35.180116 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.179902    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-out" pod "df960743-2f84-4069-8a1d-0c8e65af4d9b" (UID: "df960743-2f84-4069-8a1d-0c8e65af4d9b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:11:35.185387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.185351    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-cluster-tls-config" pod "df960743-2f84-4069-8a1d-0c8e65af4d9b" (UID: "df960743-2f84-4069-8a1d-0c8e65af4d9b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:35.192622 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.192601    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-web-config" pod "df960743-2f84-4069-8a1d-0c8e65af4d9b" (UID: "df960743-2f84-4069-8a1d-0c8e65af4d9b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:11:35.270197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270140    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8trw\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-kube-api-access-c8trw\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270163    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270173    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270182    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270191    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df960743-2f84-4069-8a1d-0c8e65af4d9b-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270199    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270207    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270216    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/98cd68cb-5d87-42af-8d7d-4f842be844b9-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270223    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98cd68cb-5d87-42af-8d7d-4f842be844b9-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270230    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df960743-2f84-4069-8a1d-0c8e65af4d9b-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270237    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqskd\" (UniqueName: \"kubernetes.io/projected/98cd68cb-5d87-42af-8d7d-4f842be844b9-kube-api-access-hqskd\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.270450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.270246    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df960743-2f84-4069-8a1d-0c8e65af4d9b-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:11:35.896815 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.896782    4849 generic.go:356] "Generic (PLEG): container finished" podID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerID="42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda" exitCode=0
Oct 30 11:11:35.896989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.896853    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:11:35.896989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.896863    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerDied","Data":"42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda"}
Oct 30 11:11:35.896989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.896944    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"df960743-2f84-4069-8a1d-0c8e65af4d9b","Type":"ContainerDied","Data":"1f9aa7485d5107d334854ce485f264660005501bc526c45fb7d4935da4a73c06"}
Oct 30 11:11:35.896989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.896986    4849 scope.go:122] "RemoveContainer" containerID="ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee"
Oct 30 11:11:35.898390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.898263    4849 generic.go:356] "Generic (PLEG): container finished" podID="98cd68cb-5d87-42af-8d7d-4f842be844b9" containerID="fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38" exitCode=0
Oct 30 11:11:35.898390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.898314    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"98cd68cb-5d87-42af-8d7d-4f842be844b9","Type":"ContainerDied","Data":"fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38"}
Oct 30 11:11:35.898390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.898320    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:35.898390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.898352    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"98cd68cb-5d87-42af-8d7d-4f842be844b9","Type":"ContainerDied","Data":"afd4fa026c373e643a7e48fd538389c9fd391275ee0aec703b17a47028be19f7"}
Oct 30 11:11:35.906625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.906608    4849 scope.go:122] "RemoveContainer" containerID="42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda"
Oct 30 11:11:35.914718 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.914701    4849 scope.go:122] "RemoveContainer" containerID="9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6"
Oct 30 11:11:35.921944 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.921925    4849 scope.go:122] "RemoveContainer" containerID="ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee"
Oct 30 11:11:35.922214 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:35.922194    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee\": container with ID starting with ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee not found: ID does not exist" containerID="ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee"
Oct 30 11:11:35.922300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.922226    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee"} err="failed to get container status \"ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee\": rpc error: code = NotFound desc = could not find container \"ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee\": container with ID starting with ad3840ed2c7e7a23f29eb7b7689b8f28afbc7a69bc494c6bdb620648f53224ee not found: ID does not exist"
Oct 30 11:11:35.922300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.922249    4849 scope.go:122] "RemoveContainer" containerID="42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda"
Oct 30 11:11:35.922488 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:35.922472    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda\": container with ID starting with 42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda not found: ID does not exist" containerID="42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda"
Oct 30 11:11:35.922567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.922495    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda"} err="failed to get container status \"42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda\": rpc error: code = NotFound desc = could not find container \"42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda\": container with ID starting with 42f9b5b7bc36ff933f0f8b290b2a0fa5e5a6570295b0894d98ff64a43cf84dda not found: ID does not exist"
Oct 30 11:11:35.922567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.922516    4849 scope.go:122] "RemoveContainer" containerID="9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6"
Oct 30 11:11:35.922726 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:35.922710    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6\": container with ID starting with 9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6 not found: ID does not exist" containerID="9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6"
Oct 30 11:11:35.922786 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.922732    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6"} err="failed to get container status \"9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6\": rpc error: code = NotFound desc = could not find container \"9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6\": container with ID starting with 9159b326e612a2a388f76e635145425c9d78971cce2f0ba5cb13f6909c802fe6 not found: ID does not exist"
Oct 30 11:11:35.922786 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.922749    4849 scope.go:122] "RemoveContainer" containerID="fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38"
Oct 30 11:11:35.930144 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.930055    4849 scope.go:122] "RemoveContainer" containerID="fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38"
Oct 30 11:11:35.930341 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:35.930320    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38\": container with ID starting with fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38 not found: ID does not exist" containerID="fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38"
Oct 30 11:11:35.930403 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.930352    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38"} err="failed to get container status \"fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38\": rpc error: code = NotFound desc = could not find container \"fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38\": container with ID starting with fdc4b55556ed1d9e5d333cdfbe4af95c216a5d66e84e45c61672c5be62f35f38 not found: ID does not exist"
Oct 30 11:11:35.952242 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.952221    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:11:35.956578 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.956560    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:11:35.994096 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.994076    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:11:35.998869 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:35.998851    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:11:36.753462 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:36.753437    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98cd68cb-5d87-42af-8d7d-4f842be844b9" path="/var/lib/kubelet/pods/98cd68cb-5d87-42af-8d7d-4f842be844b9/volumes"
Oct 30 11:11:36.753899 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:36.753882    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" path="/var/lib/kubelet/pods/df960743-2f84-4069-8a1d-0c8e65af4d9b/volumes"
Oct 30 11:11:36.754473 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:36.754458    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" path="/var/lib/kubelet/pods/ef541574-0c7d-4a58-b3fb-4cf1f820217c/volumes"
Oct 30 11:11:38.769627 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.769600    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:11:38.769974 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.769961    4849 cpu_manager.go:414] "containerMap: removing container" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="prometheus"
Oct 30 11:11:38.769974 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.769972    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="prometheus"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.769981    4849 cpu_manager.go:414] "containerMap: removing container" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="thanos-sidecar"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.769986    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="thanos-sidecar"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.769996    4849 cpu_manager.go:414] "containerMap: removing container" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="config-reloader"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770001    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="config-reloader"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770009    4849 cpu_manager.go:414] "containerMap: removing container" podUID="b5b1ffda-63a4-455d-a5ef-f6243dcf2480" containerName="insights-gathering"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770014    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5b1ffda-63a4-455d-a5ef-f6243dcf2480" containerName="insights-gathering"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770046    4849 cpu_manager.go:414] "containerMap: removing container" podUID="282f747d-fc70-463d-b2a9-683add07f4c8" containerName="curl"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770057    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="282f747d-fc70-463d-b2a9-683add07f4c8" containerName="curl"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770066    4849 cpu_manager.go:414] "containerMap: removing container" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="init-config-reloader"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770070    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="init-config-reloader"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770076    4849 cpu_manager.go:414] "containerMap: removing container" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="init-config-reloader"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770080    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="init-config-reloader"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770086    4849 cpu_manager.go:414] "containerMap: removing container" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="config-reloader"
Oct 30 11:11:38.770088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770090    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="config-reloader"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770117    4849 cpu_manager.go:414] "containerMap: removing container" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="alertmanager"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770121    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="alertmanager"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770127    4849 cpu_manager.go:414] "containerMap: removing container" podUID="98cd68cb-5d87-42af-8d7d-4f842be844b9" containerName="otc-container"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770131    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98cd68cb-5d87-42af-8d7d-4f842be844b9" containerName="otc-container"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770182    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="alertmanager"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770190    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="98cd68cb-5d87-42af-8d7d-4f842be844b9" containerName="otc-container"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770196    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="prometheus"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770202    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="thanos-sidecar"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770207    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef541574-0c7d-4a58-b3fb-4cf1f820217c" containerName="config-reloader"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770213    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5b1ffda-63a4-455d-a5ef-f6243dcf2480" containerName="insights-gathering"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770221    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="282f747d-fc70-463d-b2a9-683add07f4c8" containerName="curl"
Oct 30 11:11:38.770491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.770226    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="df960743-2f84-4069-8a1d-0c8e65af4d9b" containerName="config-reloader"
Oct 30 11:11:38.775266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.775240    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.793659 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.793637    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:11:38.798193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.798168    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d2a09985-8295-4f29-aa2f-36d410ff9d57-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.798335 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.798232    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2a09985-8295-4f29-aa2f-36d410ff9d57-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.798335 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.798286    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcs7s\" (UniqueName: \"kubernetes.io/projected/d2a09985-8295-4f29-aa2f-36d410ff9d57-kube-api-access-kcs7s\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.807069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.807041    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:11:38.807377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.807357    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-61f7caf9\""
Oct 30 11:11:38.807493 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.807451    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-8vgtc\""
Oct 30 11:11:38.807558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.807506    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:11:38.809388 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.809359    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:11:38.899322 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.899297    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d2a09985-8295-4f29-aa2f-36d410ff9d57-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.899468 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.899339    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2a09985-8295-4f29-aa2f-36d410ff9d57-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.899468 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.899356    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcs7s\" (UniqueName: \"kubernetes.io/projected/d2a09985-8295-4f29-aa2f-36d410ff9d57-kube-api-access-kcs7s\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.899982 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.899956    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d2a09985-8295-4f29-aa2f-36d410ff9d57-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.901814 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.901796    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2a09985-8295-4f29-aa2f-36d410ff9d57-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:38.923783 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:38.923756    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcs7s\" (UniqueName: \"kubernetes.io/projected/d2a09985-8295-4f29-aa2f-36d410ff9d57-kube-api-access-kcs7s\") pod \"data-science-collector-collector-0\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:39.090911 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.090888    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:11:39.349519 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.349439    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:11:39.353798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.353782    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.376516 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.376464    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 11:11:39.376636 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.376514    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-serving-cabundle\""
Oct 30 11:11:39.386705 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.386683    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-config\""
Oct 30 11:11:39.386810 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.386797    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 11:11:39.386877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.386822    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-dockercfg-hqrgs\""
Oct 30 11:11:39.386877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.386841    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway-serving-cert\""
Oct 30 11:11:39.386965 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.386950    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-mtls\""
Oct 30 11:11:39.387774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.387759    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-ca-bundle\""
Oct 30 11:11:39.388851 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.388829    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:11:39.393202 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.393183    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:11:39.396051 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:11:39.396008    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a09985_8295_4f29_aa2f_36d410ff9d57.slice/crio-72eb4eb6b99fb5526a5403b7fa34300c64911cab6372ce60144d604a5aa20ee0 WatchSource:0}: Error finding container 72eb4eb6b99fb5526a5403b7fa34300c64911cab6372ce60144d604a5aa20ee0: Status 404 returned error can't find the container with id 72eb4eb6b99fb5526a5403b7fa34300c64911cab6372ce60144d604a5aa20ee0
Oct 30 11:11:39.398314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.398281    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:11:39.398393 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.398347    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:11:39.435766 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:11:39.435734    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:11:39.504840 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.504811    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.504985 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.504955    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.505072 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.505056    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.505132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.505087    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.505132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.505110    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.505132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.505125    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.505264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.505183    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-866k9\" (UniqueName: \"kubernetes.io/projected/606e386e-4f1b-46ea-b079-bf4b41fc8908-kube-api-access-866k9\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.505264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.505220    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606299 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606227    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606305    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606346    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606379    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606410    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606432    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606712 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606457    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-866k9\" (UniqueName: \"kubernetes.io/projected/606e386e-4f1b-46ea-b079-bf4b41fc8908-kube-api-access-866k9\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.606712 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.606488    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.607043 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.607004    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.607146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.607085    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.607269 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.607240    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.607452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.607431    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.609039 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.608997    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.609298 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.609282    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.609354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.609337    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.631204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.631185    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-866k9\" (UniqueName: \"kubernetes.io/projected/606e386e-4f1b-46ea-b079-bf4b41fc8908-kube-api-access-866k9\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:39.921255 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.921184    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d2a09985-8295-4f29-aa2f-36d410ff9d57","Type":"ContainerStarted","Data":"88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57"}
Oct 30 11:11:39.921255 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.921250    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d2a09985-8295-4f29-aa2f-36d410ff9d57","Type":"ContainerStarted","Data":"72eb4eb6b99fb5526a5403b7fa34300c64911cab6372ce60144d604a5aa20ee0"}
Oct 30 11:11:39.967807 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:39.967766    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.967753397 podStartE2EDuration="1.967753397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:11:39.965802327 +0000 UTC m=+1597.346251766" watchObservedRunningTime="2026-10-30 11:11:39.967753397 +0000 UTC m=+1597.348202837"
Oct 30 11:11:46.757589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:46.757555    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:46.789016 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:46.788987    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") device path: \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:46.858290 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:46.858252    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:46.858458 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:46.858428    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") DevicePath \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:46.874191 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:46.874158    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:46.979746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:46.979716    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/fd3ed81014d76a975b5ba6f044c00684e55bfb3daddf6ff0e225ceff74c57792/globalmount\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:47.008103 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.008049    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:47.174197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.174174    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:47.509839 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:11:47.509806    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod606e386e_4f1b_46ea_b079_bf4b41fc8908.slice/crio-0be0282bdb10ad248140d610f81ca47158e68a88961ff29832dfd2237ac1f487 WatchSource:0}: Error finding container 0be0282bdb10ad248140d610f81ca47158e68a88961ff29832dfd2237ac1f487: Status 404 returned error can't find the container with id 0be0282bdb10ad248140d610f81ca47158e68a88961ff29832dfd2237ac1f487
Oct 30 11:11:47.520729 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.520703    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:11:47.963802 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.963769    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerStarted","Data":"31b4404163706af5868fee17a8af3959ea11d86ef4011c21f5a467d8ac3ce3a8"}
Oct 30 11:11:47.964239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.963833    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerStarted","Data":"755e023137119e191dae417964914ba98e216c8be7ec55f7a49bef62e5e0f031"}
Oct 30 11:11:47.964239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.963858    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerStarted","Data":"3a902367f7dcf558e1e1b9cb5737de2e837e770cd2629794ca15b76bf64bb896"}
Oct 30 11:11:47.964239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.964058    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:47.964239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.964148    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerStarted","Data":"0be0282bdb10ad248140d610f81ca47158e68a88961ff29832dfd2237ac1f487"}
Oct 30 11:11:47.964239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.964188    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:47.964239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.964214    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:47.966435 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.966413    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:47.972226 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:47.972208    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:11:48.046309 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:11:48.046267    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podStartSLOduration=9.046254132 podStartE2EDuration="9.046254132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:11:48.031766911 +0000 UTC m=+1605.412216349" watchObservedRunningTime="2026-10-30 11:11:48.046254132 +0000 UTC m=+1605.426703570"
Oct 30 11:12:09.430806 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:12:09.430765    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:12:09.976209 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:09.976168    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:12:19.975645 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:19.975602    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:12:29.976210 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:29.976169    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:32.553972 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.553940    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:32.554754 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.554206    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="d2a09985-8295-4f29-aa2f-36d410ff9d57" containerName="otc-container" containerID="cri-o://88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57" gracePeriod=30
Oct 30 11:12:32.809106 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.809083    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:32.899177 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.899151    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2a09985-8295-4f29-aa2f-36d410ff9d57-otc-internal\" (UniqueName: \"kubernetes.io/configmap/d2a09985-8295-4f29-aa2f-36d410ff9d57-otc-internal\") pod \"d2a09985-8295-4f29-aa2f-36d410ff9d57\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") "
Oct 30 11:12:32.899342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.899225    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2a09985-8295-4f29-aa2f-36d410ff9d57-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2a09985-8295-4f29-aa2f-36d410ff9d57-monitoring-tls-certs\") pod \"d2a09985-8295-4f29-aa2f-36d410ff9d57\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") "
Oct 30 11:12:32.899342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.899246    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2a09985-8295-4f29-aa2f-36d410ff9d57-kube-api-access-kcs7s\" (UniqueName: \"kubernetes.io/projected/d2a09985-8295-4f29-aa2f-36d410ff9d57-kube-api-access-kcs7s\") pod \"d2a09985-8295-4f29-aa2f-36d410ff9d57\" (UID: \"d2a09985-8295-4f29-aa2f-36d410ff9d57\") "
Oct 30 11:12:32.899563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.899538    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a09985-8295-4f29-aa2f-36d410ff9d57-otc-internal" pod "d2a09985-8295-4f29-aa2f-36d410ff9d57" (UID: "d2a09985-8295-4f29-aa2f-36d410ff9d57"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:12:32.901276 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.901244    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a09985-8295-4f29-aa2f-36d410ff9d57-monitoring-tls-certs" pod "d2a09985-8295-4f29-aa2f-36d410ff9d57" (UID: "d2a09985-8295-4f29-aa2f-36d410ff9d57"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:12:32.901362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.901313    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a09985-8295-4f29-aa2f-36d410ff9d57-kube-api-access-kcs7s" pod "d2a09985-8295-4f29-aa2f-36d410ff9d57" (UID: "d2a09985-8295-4f29-aa2f-36d410ff9d57"). InnerVolumeSpecName "kube-api-access-kcs7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:12:32.998724 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.998695    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:12:32.998949 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.998921    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo" containerID="cri-o://3a902367f7dcf558e1e1b9cb5737de2e837e770cd2629794ca15b76bf64bb896" gracePeriod=30
Oct 30 11:12:32.999092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.998944    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway" containerID="cri-o://755e023137119e191dae417964914ba98e216c8be7ec55f7a49bef62e5e0f031" gracePeriod=30
Oct 30 11:12:32.999092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:32.998930    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway-opa" containerID="cri-o://31b4404163706af5868fee17a8af3959ea11d86ef4011c21f5a467d8ac3ce3a8" gracePeriod=30
Oct 30 11:12:33.000910 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.000880    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2a09985-8295-4f29-aa2f-36d410ff9d57-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:33.001048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.000915    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcs7s\" (UniqueName: \"kubernetes.io/projected/d2a09985-8295-4f29-aa2f-36d410ff9d57-kube-api-access-kcs7s\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:33.001048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.000932    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d2a09985-8295-4f29-aa2f-36d410ff9d57-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:33.197108 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.197078    4849 generic.go:356] "Generic (PLEG): container finished" podID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerID="31b4404163706af5868fee17a8af3959ea11d86ef4011c21f5a467d8ac3ce3a8" exitCode=0
Oct 30 11:12:33.197221 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.197150    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerDied","Data":"31b4404163706af5868fee17a8af3959ea11d86ef4011c21f5a467d8ac3ce3a8"}
Oct 30 11:12:33.198425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.198402    4849 generic.go:356] "Generic (PLEG): container finished" podID="d2a09985-8295-4f29-aa2f-36d410ff9d57" containerID="88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57" exitCode=0
Oct 30 11:12:33.198534 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.198458    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:33.198534 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.198484    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d2a09985-8295-4f29-aa2f-36d410ff9d57","Type":"ContainerDied","Data":"88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57"}
Oct 30 11:12:33.198534 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.198529    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d2a09985-8295-4f29-aa2f-36d410ff9d57","Type":"ContainerDied","Data":"72eb4eb6b99fb5526a5403b7fa34300c64911cab6372ce60144d604a5aa20ee0"}
Oct 30 11:12:33.198660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.198550    4849 scope.go:122] "RemoveContainer" containerID="88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57"
Oct 30 11:12:33.212345 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.212330    4849 scope.go:122] "RemoveContainer" containerID="88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57"
Oct 30 11:12:33.212618 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:12:33.212601    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57\": container with ID starting with 88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57 not found: ID does not exist" containerID="88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57"
Oct 30 11:12:33.212679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.212626    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57"} err="failed to get container status \"88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57\": rpc error: code = NotFound desc = could not find container \"88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57\": container with ID starting with 88e48e145854b3cc6f55a9abaa7c83e618a320e4e7c1ffd62a41f1d17890bb57 not found: ID does not exist"
Oct 30 11:12:33.252972 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.252950    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:33.258475 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:33.258450    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:34.206842 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.206806    4849 generic.go:356] "Generic (PLEG): container finished" podID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerID="755e023137119e191dae417964914ba98e216c8be7ec55f7a49bef62e5e0f031" exitCode=0
Oct 30 11:12:34.206842 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.206832    4849 generic.go:356] "Generic (PLEG): container finished" podID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerID="3a902367f7dcf558e1e1b9cb5737de2e837e770cd2629794ca15b76bf64bb896" exitCode=0
Oct 30 11:12:34.207212 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.206840    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerDied","Data":"755e023137119e191dae417964914ba98e216c8be7ec55f7a49bef62e5e0f031"}
Oct 30 11:12:34.207212 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.206918    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerDied","Data":"3a902367f7dcf558e1e1b9cb5737de2e837e770cd2629794ca15b76bf64bb896"}
Oct 30 11:12:34.246704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.246683    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:34.310515 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310468    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.310615 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310555    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.310615 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310581    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-ca-bundle\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.310615 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310607    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-rbac\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.310756 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310626    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.310756 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310651    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/606e386e-4f1b-46ea-b079-bf4b41fc8908-kube-api-access-866k9\" (UniqueName: \"kubernetes.io/projected/606e386e-4f1b-46ea-b079-bf4b41fc8908-kube-api-access-866k9\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.310756 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310669    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-conf\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.310756 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310733    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-mtls\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.311225 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310788    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-tenants\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-tenants\") pod \"606e386e-4f1b-46ea-b079-bf4b41fc8908\" (UID: \"606e386e-4f1b-46ea-b079-bf4b41fc8908\") "
Oct 30 11:12:34.311225 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310960    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-ca-bundle" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:12:34.311225 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.310981    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-rbac" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "gateway-rbac". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:12:34.311225 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.311151    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.311477 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.311224    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-rbac\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.311477 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.311336    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-conf" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:12:34.311477 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.311392    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-serving-cabundle" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-serving-cabundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:12:34.313339 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.313313    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-mtls" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:12:34.313522 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.313475    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-tenants" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "gateway-tenants". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:12:34.313596 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.313567    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-gateway-serving-cert" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-gateway-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:12:34.313596 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.313585    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/606e386e-4f1b-46ea-b079-bf4b41fc8908-kube-api-access-866k9" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "kube-api-access-866k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:12:34.313956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.313938    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e" pod "606e386e-4f1b-46ea-b079-bf4b41fc8908" (UID: "606e386e-4f1b-46ea-b079-bf4b41fc8908"). InnerVolumeSpecName "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:12:34.412204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.412184    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-gateway-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.412204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.412204    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-866k9\" (UniqueName: \"kubernetes.io/projected/606e386e-4f1b-46ea-b079-bf4b41fc8908-kube-api-access-866k9\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.412333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.412213    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.412333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.412224    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.412333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.412237    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/606e386e-4f1b-46ea-b079-bf4b41fc8908-gateway-tenants\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.412333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.412250    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/606e386e-4f1b-46ea-b079-bf4b41fc8908-tempo-data-science-tempomonolithic-serving-cabundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:34.412333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.412275    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:12:34.420849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.420834    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:12:34.513279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.513259    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\""
Oct 30 11:12:34.753007 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:34.752980    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2a09985-8295-4f29-aa2f-36d410ff9d57" path="/var/lib/kubelet/pods/d2a09985-8295-4f29-aa2f-36d410ff9d57/volumes"
Oct 30 11:12:35.213212 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.213184    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:35.213212 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.213197    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"606e386e-4f1b-46ea-b079-bf4b41fc8908","Type":"ContainerDied","Data":"0be0282bdb10ad248140d610f81ca47158e68a88961ff29832dfd2237ac1f487"}
Oct 30 11:12:35.213708 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.213261    4849 scope.go:122] "RemoveContainer" containerID="31b4404163706af5868fee17a8af3959ea11d86ef4011c21f5a467d8ac3ce3a8"
Oct 30 11:12:35.221496 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.221478    4849 scope.go:122] "RemoveContainer" containerID="755e023137119e191dae417964914ba98e216c8be7ec55f7a49bef62e5e0f031"
Oct 30 11:12:35.229003 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.228988    4849 scope.go:122] "RemoveContainer" containerID="3a902367f7dcf558e1e1b9cb5737de2e837e770cd2629794ca15b76bf64bb896"
Oct 30 11:12:35.282706 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.282685    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:12:35.296180 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.296159    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:12:35.741493 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.741457    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:35.741998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.741979    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d2a09985-8295-4f29-aa2f-36d410ff9d57" containerName="otc-container"
Oct 30 11:12:35.742053 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742001    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2a09985-8295-4f29-aa2f-36d410ff9d57" containerName="otc-container"
Oct 30 11:12:35.742117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742054    4849 cpu_manager.go:414] "containerMap: removing container" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway-opa"
Oct 30 11:12:35.742117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742064    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway-opa"
Oct 30 11:12:35.742117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742087    4849 cpu_manager.go:414] "containerMap: removing container" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo"
Oct 30 11:12:35.742117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742095    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo"
Oct 30 11:12:35.742117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742105    4849 cpu_manager.go:414] "containerMap: removing container" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway"
Oct 30 11:12:35.742117 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742112    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway"
Oct 30 11:12:35.742293 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742215    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2a09985-8295-4f29-aa2f-36d410ff9d57" containerName="otc-container"
Oct 30 11:12:35.742293 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742234    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway-opa"
Oct 30 11:12:35.742293 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742246    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo"
Oct 30 11:12:35.742293 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.742254    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" containerName="tempo-gateway"
Oct 30 11:12:35.747344 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.747317    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.788707 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.788680    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:12:35.788707 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.788701    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:12:35.788878 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.788751    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:12:35.788878 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.788829    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-61f7caf9\""
Oct 30 11:12:35.788954 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.788890    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-rn48g\""
Oct 30 11:12:35.791381 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.791356    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:35.821371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.821341    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/914b283b-b7b2-47d5-a0ff-9c6059658027-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.821500 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.821482    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/914b283b-b7b2-47d5-a0ff-9c6059658027-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.821556 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.821526    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjftm\" (UniqueName: \"kubernetes.io/projected/914b283b-b7b2-47d5-a0ff-9c6059658027-kube-api-access-vjftm\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.922660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.922631    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/914b283b-b7b2-47d5-a0ff-9c6059658027-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.922765 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.922685    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/914b283b-b7b2-47d5-a0ff-9c6059658027-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.922821 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.922800    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjftm\" (UniqueName: \"kubernetes.io/projected/914b283b-b7b2-47d5-a0ff-9c6059658027-kube-api-access-vjftm\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.923308 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.923287    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/914b283b-b7b2-47d5-a0ff-9c6059658027-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.925256 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.925234    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/914b283b-b7b2-47d5-a0ff-9c6059658027-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:35.946917 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:35.946893    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjftm\" (UniqueName: \"kubernetes.io/projected/914b283b-b7b2-47d5-a0ff-9c6059658027-kube-api-access-vjftm\") pod \"data-science-collector-collector-0\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:36.061882 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:36.061861    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:36.363518 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:36.363453    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:36.366682 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:12:36.366655    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914b283b_b7b2_47d5_a0ff_9c6059658027.slice/crio-04e7df56686984d1e3d6d94105ab2b104fd991565ad47dcbc19a849e5a26c8f1 WatchSource:0}: Error finding container 04e7df56686984d1e3d6d94105ab2b104fd991565ad47dcbc19a849e5a26c8f1: Status 404 returned error can't find the container with id 04e7df56686984d1e3d6d94105ab2b104fd991565ad47dcbc19a849e5a26c8f1
Oct 30 11:12:36.368628 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:36.368602    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:12:36.368695 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:36.368653    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:12:36.753317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:36.753243    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="606e386e-4f1b-46ea-b079-bf4b41fc8908" path="/var/lib/kubelet/pods/606e386e-4f1b-46ea-b079-bf4b41fc8908/volumes"
Oct 30 11:12:37.226433 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:37.226395    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"914b283b-b7b2-47d5-a0ff-9c6059658027","Type":"ContainerStarted","Data":"6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481"}
Oct 30 11:12:37.226627 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:37.226455    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"914b283b-b7b2-47d5-a0ff-9c6059658027","Type":"ContainerStarted","Data":"04e7df56686984d1e3d6d94105ab2b104fd991565ad47dcbc19a849e5a26c8f1"}
Oct 30 11:12:37.271018 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:37.270967    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=2.270954186 podStartE2EDuration="2.270954186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:12:37.269580709 +0000 UTC m=+1654.650030150" watchObservedRunningTime="2026-10-30 11:12:37.270954186 +0000 UTC m=+1654.651403625"
Oct 30 11:12:39.430381 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:12:39.430351    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:12:51.390386 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.390356    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:51.390893 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.390572    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="914b283b-b7b2-47d5-a0ff-9c6059658027" containerName="otc-container" containerID="cri-o://6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481" gracePeriod=30
Oct 30 11:12:51.640456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.640406    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:51.747372 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.747345    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/914b283b-b7b2-47d5-a0ff-9c6059658027-kube-api-access-vjftm\" (UniqueName: \"kubernetes.io/projected/914b283b-b7b2-47d5-a0ff-9c6059658027-kube-api-access-vjftm\") pod \"914b283b-b7b2-47d5-a0ff-9c6059658027\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") "
Oct 30 11:12:51.747543 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.747471    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/914b283b-b7b2-47d5-a0ff-9c6059658027-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/914b283b-b7b2-47d5-a0ff-9c6059658027-monitoring-tls-certs\") pod \"914b283b-b7b2-47d5-a0ff-9c6059658027\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") "
Oct 30 11:12:51.747543 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.747511    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/914b283b-b7b2-47d5-a0ff-9c6059658027-otc-internal\" (UniqueName: \"kubernetes.io/configmap/914b283b-b7b2-47d5-a0ff-9c6059658027-otc-internal\") pod \"914b283b-b7b2-47d5-a0ff-9c6059658027\" (UID: \"914b283b-b7b2-47d5-a0ff-9c6059658027\") "
Oct 30 11:12:51.747858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.747832    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914b283b-b7b2-47d5-a0ff-9c6059658027-otc-internal" pod "914b283b-b7b2-47d5-a0ff-9c6059658027" (UID: "914b283b-b7b2-47d5-a0ff-9c6059658027"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:12:51.749371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.749344    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914b283b-b7b2-47d5-a0ff-9c6059658027-monitoring-tls-certs" pod "914b283b-b7b2-47d5-a0ff-9c6059658027" (UID: "914b283b-b7b2-47d5-a0ff-9c6059658027"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:12:51.749475 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.749410    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914b283b-b7b2-47d5-a0ff-9c6059658027-kube-api-access-vjftm" pod "914b283b-b7b2-47d5-a0ff-9c6059658027" (UID: "914b283b-b7b2-47d5-a0ff-9c6059658027"). InnerVolumeSpecName "kube-api-access-vjftm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:12:51.848625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.848598    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/914b283b-b7b2-47d5-a0ff-9c6059658027-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:51.848625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.848623    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/914b283b-b7b2-47d5-a0ff-9c6059658027-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:51.848801 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:51.848633    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjftm\" (UniqueName: \"kubernetes.io/projected/914b283b-b7b2-47d5-a0ff-9c6059658027-kube-api-access-vjftm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:12:52.310132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.310091    4849 generic.go:356] "Generic (PLEG): container finished" podID="914b283b-b7b2-47d5-a0ff-9c6059658027" containerID="6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481" exitCode=0
Oct 30 11:12:52.310342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.310141    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"914b283b-b7b2-47d5-a0ff-9c6059658027","Type":"ContainerDied","Data":"6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481"}
Oct 30 11:12:52.310342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.310155    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:52.310342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.310191    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"914b283b-b7b2-47d5-a0ff-9c6059658027","Type":"ContainerDied","Data":"04e7df56686984d1e3d6d94105ab2b104fd991565ad47dcbc19a849e5a26c8f1"}
Oct 30 11:12:52.310342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.310214    4849 scope.go:122] "RemoveContainer" containerID="6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481"
Oct 30 11:12:52.322064 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.322040    4849 scope.go:122] "RemoveContainer" containerID="6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481"
Oct 30 11:12:52.322371 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:12:52.322349    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481\": container with ID starting with 6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481 not found: ID does not exist" containerID="6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481"
Oct 30 11:12:52.322451 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.322383    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481"} err="failed to get container status \"6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481\": rpc error: code = NotFound desc = could not find container \"6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481\": container with ID starting with 6aeedac85c6dcd6ac52c659b6626ab79cb645bf39ba3ec3cab5825551ebb9481 not found: ID does not exist"
Oct 30 11:12:52.365942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.365912    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:52.370043 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.370010    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:52.753205 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:52.753148    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="914b283b-b7b2-47d5-a0ff-9c6059658027" path="/var/lib/kubelet/pods/914b283b-b7b2-47d5-a0ff-9c6059658027/volumes"
Oct 30 11:12:54.564312 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.564279    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:54.564704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.564600    4849 cpu_manager.go:414] "containerMap: removing container" podUID="914b283b-b7b2-47d5-a0ff-9c6059658027" containerName="otc-container"
Oct 30 11:12:54.564704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.564616    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="914b283b-b7b2-47d5-a0ff-9c6059658027" containerName="otc-container"
Oct 30 11:12:54.564783 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.564754    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="914b283b-b7b2-47d5-a0ff-9c6059658027" containerName="otc-container"
Oct 30 11:12:54.567704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.567688    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.588278 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.588251    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:12:54.598919 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.598893    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:12:54.599069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.598928    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:12:54.599069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.598974    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-61f7caf9\""
Oct 30 11:12:54.600932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.600910    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:54.601405 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.601379    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-vdrlt\""
Oct 30 11:12:54.671949 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.671925    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.672062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.671964    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69fv5\" (UniqueName: \"kubernetes.io/projected/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-kube-api-access-69fv5\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.672113 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.672104    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.773446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.773418    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.773567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.773450    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.773567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.773475    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69fv5\" (UniqueName: \"kubernetes.io/projected/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-kube-api-access-69fv5\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.774302 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.774274    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.775967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.775944    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.799394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.799370    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69fv5\" (UniqueName: \"kubernetes.io/projected/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-kube-api-access-69fv5\") pod \"data-science-collector-collector-0\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:54.888180 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:54.888125    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:12:55.192681 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.192659    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:12:55.195008 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:12:55.194980    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b31fef2_8f7d_4da9_ace6_12b060e8d7cd.slice/crio-983a3d45ead449bba68a383396e1963dddb8ff7a457358c9cc3d6f83d7da04e6 WatchSource:0}: Error finding container 983a3d45ead449bba68a383396e1963dddb8ff7a457358c9cc3d6f83d7da04e6: Status 404 returned error can't find the container with id 983a3d45ead449bba68a383396e1963dddb8ff7a457358c9cc3d6f83d7da04e6
Oct 30 11:12:55.197467 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.197424    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:12:55.197550 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.197508    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:12:55.333092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.333043    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd","Type":"ContainerStarted","Data":"dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27"}
Oct 30 11:12:55.333243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.333178    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd","Type":"ContainerStarted","Data":"983a3d45ead449bba68a383396e1963dddb8ff7a457358c9cc3d6f83d7da04e6"}
Oct 30 11:12:55.380994 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.380945    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.380932176 podStartE2EDuration="1.380932176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:12:55.379919178 +0000 UTC m=+1672.760368617" watchObservedRunningTime="2026-10-30 11:12:55.380932176 +0000 UTC m=+1672.761381615"
Oct 30 11:12:55.857706 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.857667    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:12:55.861575 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.861552    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.879037 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.879002    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 11:12:55.879181 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.879165    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-mtls\""
Oct 30 11:12:55.879248 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.879233    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-dockercfg-kkjwt\""
Oct 30 11:12:55.879485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.879470    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway\""
Oct 30 11:12:55.879614 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.879599    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-config\""
Oct 30 11:12:55.879677 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.879655    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-ca-bundle\""
Oct 30 11:12:55.891272 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.891250    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-serving-cabundle\""
Oct 30 11:12:55.893421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.893371    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:12:55.979213 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.979191    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-gateway-serving-cert\""
Oct 30 11:12:55.985116 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985096    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.985197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985131    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.985197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985156    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.985279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985215    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.985316 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985277    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.985357 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985314    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.985357 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985333    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk4fv\" (UniqueName: \"kubernetes.io/projected/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-kube-api-access-xk4fv\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:55.985444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:55.985410    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.085833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.085805    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.085950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.085844    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.085950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.085878    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.086096 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.086000    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.086096 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.086033    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk4fv\" (UniqueName: \"kubernetes.io/projected/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-kube-api-access-xk4fv\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.086265 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.086242    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.086346 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.086324    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.086423 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.086392    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.086762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.086736    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.087115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.087070    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-rbac\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.087115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.087109    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-conf\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.087261 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.087197    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-ca-bundle\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.089098 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.089072    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-tenants\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.089098 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.089089    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.089275 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.089254    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-mtls\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:12:56.110531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:12:56.110482    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk4fv\" (UniqueName: \"kubernetes.io/projected/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-kube-api-access-xk4fv\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:03.060233 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:03.060199    4849 scope.go:122] "RemoveContainer" containerID="309dcddef4f778a9cbb44be3bda4c8a6cb12e516f831482164581822cb035770"
Oct 30 11:13:04.750344 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:04.750313    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:04.782125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:04.782101    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") device path: \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:04.850971 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:04.850943    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:04.851085 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:04.851062    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") DevicePath \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:04.867303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:04.867280    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:04.970496 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:04.970467    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/fd3ed81014d76a975b5ba6f044c00684e55bfb3daddf6ff0e225ceff74c57792/globalmount\"" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:04.998163 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:04.998144    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"tempo-data-science-tempomonolithic-0\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") " pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:05.195200 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:05.195178    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempomonolithic-dockercfg-kkjwt\""
Oct 30 11:13:05.200239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:05.200222    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:05.534350 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:05.534323    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:13:05.536004 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:13:05.535977    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51694aa6_8abc_4177_9da6_8a2d6e00c6ba.slice/crio-5ec5b6968471e0f01eb94b8925e983b6c4a896bb929ffa36983401693ffe7a9e WatchSource:0}: Error finding container 5ec5b6968471e0f01eb94b8925e983b6c4a896bb929ffa36983401693ffe7a9e: Status 404 returned error can't find the container with id 5ec5b6968471e0f01eb94b8925e983b6c4a896bb929ffa36983401693ffe7a9e
Oct 30 11:13:06.392109 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.392069    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerStarted","Data":"de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3"}
Oct 30 11:13:06.392567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.392121    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerStarted","Data":"78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176"}
Oct 30 11:13:06.392567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.392138    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerStarted","Data":"10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80"}
Oct 30 11:13:06.392567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.392227    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:06.392567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.392283    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerStarted","Data":"5ec5b6968471e0f01eb94b8925e983b6c4a896bb929ffa36983401693ffe7a9e"}
Oct 30 11:13:06.392567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.392322    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:06.392567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.392366    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:06.394986 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.394963    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:06.401433 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.401410    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:06.469991 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:06.469943    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podStartSLOduration=11.46993234 podStartE2EDuration="11.46993234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:13:06.454656878 +0000 UTC m=+1683.835106317" watchObservedRunningTime="2026-10-30 11:13:06.46993234 +0000 UTC m=+1683.850381817"
Oct 30 11:13:09.431453 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:13:09.431419    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:13:28.405541 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:28.405491    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:13:38.405151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:38.405078    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:13:39.431015 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:13:39.430982    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:13:48.405010 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:48.404966    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:50.577777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.577744    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:13:50.578192 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.578047    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" containerName="otc-container" containerID="cri-o://dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27" gracePeriod=30
Oct 30 11:13:50.841555 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.841504    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:13:50.891196 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.891171    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-monitoring-tls-certs\") pod \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") "
Oct 30 11:13:50.891336 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.891260    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-otc-internal\") pod \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") "
Oct 30 11:13:50.891336 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.891316    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-kube-api-access-69fv5\" (UniqueName: \"kubernetes.io/projected/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-kube-api-access-69fv5\") pod \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\" (UID: \"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd\") "
Oct 30 11:13:50.891681 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.891642    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-otc-internal" pod "5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" (UID: "5b31fef2-8f7d-4da9-ace6-12b060e8d7cd"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:13:50.893889 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.893863    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-kube-api-access-69fv5" pod "5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" (UID: "5b31fef2-8f7d-4da9-ace6-12b060e8d7cd"). InnerVolumeSpecName "kube-api-access-69fv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:13:50.893985 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.893868    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-monitoring-tls-certs" pod "5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" (UID: "5b31fef2-8f7d-4da9-ace6-12b060e8d7cd"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:13:50.992703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.992673    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:50.992703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.992698    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:50.992854 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:50.992709    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69fv5\" (UniqueName: \"kubernetes.io/projected/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd-kube-api-access-69fv5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:51.041992 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.041950    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:13:51.042282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.042256    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo" containerID="cri-o://10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80" gracePeriod=30
Oct 30 11:13:51.042452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.042416    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway-opa" containerID="cri-o://de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3" gracePeriod=30
Oct 30 11:13:51.042452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.042449    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway" containerID="cri-o://78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176" gracePeriod=30
Oct 30 11:13:51.392596 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.392560    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway-opa" probeResult="failure" output="Get \"http://10.132.0.72:8083/ready\": dial tcp 10.132.0.72:8083: connect: connection refused"
Oct 30 11:13:51.396444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.396409    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:13:51.637975 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.637947    4849 generic.go:356] "Generic (PLEG): container finished" podID="5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" containerID="dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27" exitCode=0
Oct 30 11:13:51.638361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.638038    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:13:51.638361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.638044    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd","Type":"ContainerDied","Data":"dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27"}
Oct 30 11:13:51.638361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.638096    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b31fef2-8f7d-4da9-ace6-12b060e8d7cd","Type":"ContainerDied","Data":"983a3d45ead449bba68a383396e1963dddb8ff7a457358c9cc3d6f83d7da04e6"}
Oct 30 11:13:51.638361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.638124    4849 scope.go:122] "RemoveContainer" containerID="dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27"
Oct 30 11:13:51.640607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.640586    4849 generic.go:356] "Generic (PLEG): container finished" podID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerID="de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3" exitCode=0
Oct 30 11:13:51.640607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.640607    4849 generic.go:356] "Generic (PLEG): container finished" podID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerID="10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80" exitCode=0
Oct 30 11:13:51.640781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.640632    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerDied","Data":"de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3"}
Oct 30 11:13:51.640781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.640668    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerDied","Data":"10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80"}
Oct 30 11:13:51.646660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.646632    4849 scope.go:122] "RemoveContainer" containerID="dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27"
Oct 30 11:13:51.646908 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:13:51.646888    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27\": container with ID starting with dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27 not found: ID does not exist" containerID="dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27"
Oct 30 11:13:51.646960 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.646916    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27"} err="failed to get container status \"dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27\": rpc error: code = NotFound desc = could not find container \"dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27\": container with ID starting with dd1e34b28d6f8424ecfffdefdb71716a81e202c979df64af7c93914693496d27 not found: ID does not exist"
Oct 30 11:13:51.692964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.692937    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:13:51.706279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:51.706258    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:13:52.306856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.306833    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:52.403680 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403626    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-gateway-serving-cert\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.403803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403688    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-ca-bundle\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.403803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403719    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-kube-api-access-xk4fv\" (UniqueName: \"kubernetes.io/projected/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-kube-api-access-xk4fv\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.403803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403761    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-rbac\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.403803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403796    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-serving-cabundle\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.404008 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403826    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-conf\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.404008 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403880    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-mtls\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.404008 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.403907    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-tenants\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-tenants\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.404203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.404013    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") pod \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\" (UID: \"51694aa6-8abc-4177-9da6-8a2d6e00c6ba\") "
Oct 30 11:13:52.404203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.404119    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-ca-bundle" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:13:52.404203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.404191    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-rbac" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "gateway-rbac". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:13:52.404368 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.404331    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.404368 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.404351    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-rbac\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-rbac\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.404876 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.404807    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-conf" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:13:52.404876 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.404810    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-serving-cabundle" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-serving-cabundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:13:52.406116 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.406078    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-gateway-serving-cert" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-gateway-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:13:52.406406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.406377    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-kube-api-access-xk4fv" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "kube-api-access-xk4fv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:13:52.406512 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.406454    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-mtls" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "tempo-data-science-tempomonolithic-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:13:52.406580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.406535    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-tenants" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "gateway-tenants". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:13:52.407421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.407398    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e" pod "51694aa6-8abc-4177-9da6-8a2d6e00c6ba" (UID: "51694aa6-8abc-4177-9da6-8a2d6e00c6ba"). InnerVolumeSpecName "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:13:52.505237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.505215    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-serving-cabundle\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-serving-cabundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.505237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.505237    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.505361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.505248    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-mtls\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.505361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.505257    4849 reconciler_common.go:299] "Volume detached for volume \"gateway-tenants\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-gateway-tenants\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.505361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.505280    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:13:52.505361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.505290    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempomonolithic-gateway-serving-cert\" (UniqueName: \"kubernetes.io/secret/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-tempo-data-science-tempomonolithic-gateway-serving-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.505361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.505299    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk4fv\" (UniqueName: \"kubernetes.io/projected/51694aa6-8abc-4177-9da6-8a2d6e00c6ba-kube-api-access-xk4fv\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:13:52.513357 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.513343    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:13:52.606346 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.606323    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-9ed1a45c-662a-4ce3-b563-4ea35a995f32\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0851a5e725334572e\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-793107b0abfd18511bd9ebaf53c2419c1a47af01a4588d7f076e42b6584c592f\""
Oct 30 11:13:52.647386 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.647364    4849 generic.go:356] "Generic (PLEG): container finished" podID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerID="78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176" exitCode=0
Oct 30 11:13:52.647715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.647418    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"
Oct 30 11:13:52.647715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.647455    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerDied","Data":"78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176"}
Oct 30 11:13:52.647715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.647524    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempomonolithic-0" event={"ID":"51694aa6-8abc-4177-9da6-8a2d6e00c6ba","Type":"ContainerDied","Data":"5ec5b6968471e0f01eb94b8925e983b6c4a896bb929ffa36983401693ffe7a9e"}
Oct 30 11:13:52.647715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.647562    4849 scope.go:122] "RemoveContainer" containerID="de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3"
Oct 30 11:13:52.656417 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.656398    4849 scope.go:122] "RemoveContainer" containerID="78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176"
Oct 30 11:13:52.664375 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.664360    4849 scope.go:122] "RemoveContainer" containerID="10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80"
Oct 30 11:13:52.671618 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.671602    4849 scope.go:122] "RemoveContainer" containerID="de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3"
Oct 30 11:13:52.671897 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:13:52.671870    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3\": container with ID starting with de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3 not found: ID does not exist" containerID="de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3"
Oct 30 11:13:52.671956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.671904    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3"} err="failed to get container status \"de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3\": rpc error: code = NotFound desc = could not find container \"de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3\": container with ID starting with de12ee97d17e1226dbbe803c4b6d7e5857e70e0ac5ef8ce825aa25ad49ffb9e3 not found: ID does not exist"
Oct 30 11:13:52.671956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.671919    4849 scope.go:122] "RemoveContainer" containerID="78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176"
Oct 30 11:13:52.672191 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:13:52.672172    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176\": container with ID starting with 78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176 not found: ID does not exist" containerID="78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176"
Oct 30 11:13:52.672247 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.672197    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176"} err="failed to get container status \"78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176\": rpc error: code = NotFound desc = could not find container \"78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176\": container with ID starting with 78aa3fccf60d04d110c31ad79e1df879b7109c8866f05f5c4831cf2a781bf176 not found: ID does not exist"
Oct 30 11:13:52.672247 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.672213    4849 scope.go:122] "RemoveContainer" containerID="10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80"
Oct 30 11:13:52.672444 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:13:52.672423    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80\": container with ID starting with 10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80 not found: ID does not exist" containerID="10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80"
Oct 30 11:13:52.672539 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.672444    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80"} err="failed to get container status \"10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80\": rpc error: code = NotFound desc = could not find container \"10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80\": container with ID starting with 10d8bcfa527d1f4fda9ea2dc1a690d6a0107ebfd36ebb87d644ab24080102e80 not found: ID does not exist"
Oct 30 11:13:52.703062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.703039    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:13:52.716647 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.716624    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempomonolithic-0"]
Oct 30 11:13:52.752257 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.752234    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" path="/var/lib/kubelet/pods/51694aa6-8abc-4177-9da6-8a2d6e00c6ba/volumes"
Oct 30 11:13:52.752692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:13:52.752679    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" path="/var/lib/kubelet/pods/5b31fef2-8f7d-4da9-ace6-12b060e8d7cd/volumes"
Oct 30 11:14:09.431453 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:14:09.431413    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:14:18.348866 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.348782    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349235    4849 cpu_manager.go:414] "containerMap: removing container" podUID="5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" containerName="otc-container"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349250    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" containerName="otc-container"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349262    4849 cpu_manager.go:414] "containerMap: removing container" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349268    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349282    4849 cpu_manager.go:414] "containerMap: removing container" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349287    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349294    4849 cpu_manager.go:414] "containerMap: removing container" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway-opa"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349299    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway-opa"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349358    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo"
Oct 30 11:14:18.349356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349365    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway-opa"
Oct 30 11:14:18.349905 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349375    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b31fef2-8f7d-4da9-ace6-12b060e8d7cd" containerName="otc-container"
Oct 30 11:14:18.349905 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.349381    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="51694aa6-8abc-4177-9da6-8a2d6e00c6ba" containerName="tempo-gateway"
Oct 30 11:14:18.354145 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.354125    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.375743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.375719    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:14:18.375743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.375742    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-0ed87044\""
Oct 30 11:14:18.376288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.376267    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:14:18.389359 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.389337    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:14:18.391487 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.391465    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:14:18.497625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.497592    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d75010c3-33e0-4ce1-9a76-7fad7acda61f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.497777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.497633    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkcxp\" (UniqueName: \"kubernetes.io/projected/d75010c3-33e0-4ce1-9a76-7fad7acda61f-kube-api-access-wkcxp\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.497777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.497745    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d75010c3-33e0-4ce1-9a76-7fad7acda61f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.599084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.599000    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d75010c3-33e0-4ce1-9a76-7fad7acda61f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.599199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.599087    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d75010c3-33e0-4ce1-9a76-7fad7acda61f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.599199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.599117    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkcxp\" (UniqueName: \"kubernetes.io/projected/d75010c3-33e0-4ce1-9a76-7fad7acda61f-kube-api-access-wkcxp\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.599911 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.599884    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d75010c3-33e0-4ce1-9a76-7fad7acda61f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.601599 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.601577    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d75010c3-33e0-4ce1-9a76-7fad7acda61f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.623696 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.623667    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkcxp\" (UniqueName: \"kubernetes.io/projected/d75010c3-33e0-4ce1-9a76-7fad7acda61f-kube-api-access-wkcxp\") pod \"data-science-collector-collector-0\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.670633 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.670612    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:14:18.976620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.976598    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:14:18.978988 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:14:18.978957    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd75010c3_33e0_4ce1_9a76_7fad7acda61f.slice/crio-6d897a5084d72e5a48608e563fa4f1aed16aeeea89148ae2284e467e94db982c WatchSource:0}: Error finding container 6d897a5084d72e5a48608e563fa4f1aed16aeeea89148ae2284e467e94db982c: Status 404 returned error can't find the container with id 6d897a5084d72e5a48608e563fa4f1aed16aeeea89148ae2284e467e94db982c
Oct 30 11:14:18.981574 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.981543    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:14:18.981690 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:18.981605    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:14:19.785612 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:19.785580    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d75010c3-33e0-4ce1-9a76-7fad7acda61f","Type":"ContainerStarted","Data":"9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab"}
Oct 30 11:14:19.786036 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:19.785627    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d75010c3-33e0-4ce1-9a76-7fad7acda61f","Type":"ContainerStarted","Data":"6d897a5084d72e5a48608e563fa4f1aed16aeeea89148ae2284e467e94db982c"}
Oct 30 11:14:19.831074 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:19.831012    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.831000196 podStartE2EDuration="1.831000196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:14:19.829359915 +0000 UTC m=+1757.209809358" watchObservedRunningTime="2026-10-30 11:14:19.831000196 +0000 UTC m=+1757.211449635"
Oct 30 11:14:21.280119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.280083    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"]
Oct 30 11:14:21.283533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.283510    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.303109 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.303084    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-dockercfg-fhk88\""
Oct 30 11:14:21.307450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.307424    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-distributor-mtls\""
Oct 30 11:14:21.318179 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.318160    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack\""
Oct 30 11:14:21.318305 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.318188    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-ca-bundle\""
Oct 30 11:14:21.320992 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.320972    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"]
Oct 30 11:14:21.419346 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.419318    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.419456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.419349    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.419456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.419381    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h4mg\" (UniqueName: \"kubernetes.io/projected/0c4a2809-031b-4aec-a51b-9c10d17c91a2-kube-api-access-7h4mg\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.419539 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.419495    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-conf\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.419578 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.419553    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-distributor-mtls\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.466992 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.466963    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"]
Oct 30 11:14:21.470520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.470501    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.492226 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.492202    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"]
Oct 30 11:14:21.520446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.520412    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-distributor-mtls\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.520576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.520479    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.520576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.520508    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.520576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.520552    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4mg\" (UniqueName: \"kubernetes.io/projected/0c4a2809-031b-4aec-a51b-9c10d17c91a2-kube-api-access-7h4mg\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.520714 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.520593    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-conf\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.521075 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.521051    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.521372 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.521350    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.521908 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.521792    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-conf\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.523198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.523173    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-distributor-mtls\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.536938 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.536888    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"s3-secret\""
Oct 30 11:14:21.537088 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.537072    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-querier-mtls\""
Oct 30 11:14:21.545193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.545176    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h4mg\" (UniqueName: \"kubernetes.io/projected/0c4a2809-031b-4aec-a51b-9c10d17c91a2-kube-api-access-7h4mg\") pod \"tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.596801 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.596772    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:21.621934 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.621912    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.622082 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.621953    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.622082 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.621984    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pfxw\" (UniqueName: \"kubernetes.io/projected/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-kube-api-access-5pfxw\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.622188 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.622076    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-querier-mtls\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.622188 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.622128    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-conf\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.722882 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.722851    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.722996 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.722899    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pfxw\" (UniqueName: \"kubernetes.io/projected/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-kube-api-access-5pfxw\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.722996 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.722940    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-querier-mtls\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.722996 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.722959    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-conf\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.723152 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.723003    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.723432 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.723411    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.723547 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.723528    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.723815 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.723796    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-conf\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.725299 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.725278    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-querier-mtls\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.748533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.748509    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pfxw\" (UniqueName: \"kubernetes.io/projected/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-kube-api-access-5pfxw\") pod \"tempo-data-science-tempostack-querier-59c88557df-tc7qf\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.783322 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.783302    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:21.914938 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:21.914908    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"]
Oct 30 11:14:21.917639 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:14:21.917615    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4a2809_031b_4aec_a51b_9c10d17c91a2.slice/crio-e62acde65b9fc1bfff483097b6fd748700f0519ecb4830c5e594aaec0cfa6c24 WatchSource:0}: Error finding container e62acde65b9fc1bfff483097b6fd748700f0519ecb4830c5e594aaec0cfa6c24: Status 404 returned error can't find the container with id e62acde65b9fc1bfff483097b6fd748700f0519ecb4830c5e594aaec0cfa6c24
Oct 30 11:14:22.091212 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.091168    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"]
Oct 30 11:14:22.802794 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.802755    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx" event={"ID":"0c4a2809-031b-4aec-a51b-9c10d17c91a2","Type":"ContainerStarted","Data":"be588b0395f9a0187c3199c36d584f1c77d17a137c0fcf1b9fd3315e82dad20c"}
Oct 30 11:14:22.803264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.802811    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx" event={"ID":"0c4a2809-031b-4aec-a51b-9c10d17c91a2","Type":"ContainerStarted","Data":"e62acde65b9fc1bfff483097b6fd748700f0519ecb4830c5e594aaec0cfa6c24"}
Oct 30 11:14:22.803264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.802936    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:22.804356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.804332    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" event={"ID":"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7","Type":"ContainerStarted","Data":"8671bc17a4b49dac9f4bc1cbdbdae4927753d982b7f0f7dc7e8f14634feb518d"}
Oct 30 11:14:22.804429 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.804377    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" event={"ID":"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7","Type":"ContainerStarted","Data":"c9f9a6419a844e6ed34a2d35a454223825749231045f9ca8d8ceb1c7791ef180"}
Oct 30 11:14:22.804485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.804436    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:22.848165 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.848110    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx" podStartSLOduration=1.848095234 podStartE2EDuration="1.848095234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:14:22.847392657 +0000 UTC m=+1760.227842095" watchObservedRunningTime="2026-10-30 11:14:22.848095234 +0000 UTC m=+1760.228544675"
Oct 30 11:14:22.893636 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:22.893595    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" podStartSLOduration=1.8935845150000001 podStartE2EDuration="1.893584515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:14:22.891492846 +0000 UTC m=+1760.271942408" watchObservedRunningTime="2026-10-30 11:14:22.893584515 +0000 UTC m=+1760.274033954"
Oct 30 11:14:24.357175 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.357143    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:14:24.360863 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.360846    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.377981 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.377960    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-ingester-mtls\""
Oct 30 11:14:24.392915 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.392892    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:14:24.444166 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.444141    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-conf\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.444281 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.444177    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btj9h\" (UniqueName: \"kubernetes.io/projected/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-kube-api-access-btj9h\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.444281 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.444210    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.444406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.444364    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ingester-mtls\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.545418 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.545386    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btj9h\" (UniqueName: \"kubernetes.io/projected/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-kube-api-access-btj9h\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.545508 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.545434    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.545508 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.545472    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ingester-mtls\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.545508 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.545502    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-conf\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.546198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.546176    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.546337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.546317    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-conf\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.547891 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.547871    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ingester-mtls\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:24.569944 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:24.569921    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btj9h\" (UniqueName: \"kubernetes.io/projected/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-kube-api-access-btj9h\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:30.690634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:30.690594    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:30.722461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:30.722434    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") device path: \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:30.791137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:30.791104    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:30.791315 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:30.791290    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") DevicePath \"\"" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:30.807225 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:30.807200    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") DevicePath \"csi-7f600dab7f8bdfb34606269590de3813f736c88298ef584d5dfaa352a938aaad\"" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:31.044598 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.044575    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/d70e14611a39016552cec14e417282f8e35c0e121c562713dc7ac62abf656297/globalmount\"" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:31.069818 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.069795    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:31.273708 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.273688    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:31.593109 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.593084    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:14:31.595532 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:14:31.595502    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb880b417_ad1b_4c9f_85ad_6946b3ff76f9.slice/crio-c1f9f8cd840420f84f39b5eadf3198413f271bd6f66004b76c2fa4b058ba8a14 WatchSource:0}: Error finding container c1f9f8cd840420f84f39b5eadf3198413f271bd6f66004b76c2fa4b058ba8a14: Status 404 returned error can't find the container with id c1f9f8cd840420f84f39b5eadf3198413f271bd6f66004b76c2fa4b058ba8a14
Oct 30 11:14:31.851172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.851105    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" event={"ID":"b880b417-ad1b-4c9f-85ad-6946b3ff76f9","Type":"ContainerStarted","Data":"f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163"}
Oct 30 11:14:31.851172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.851152    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" event={"ID":"b880b417-ad1b-4c9f-85ad-6946b3ff76f9","Type":"ContainerStarted","Data":"c1f9f8cd840420f84f39b5eadf3198413f271bd6f66004b76c2fa4b058ba8a14"}
Oct 30 11:14:31.851555 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.851206    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:14:31.897181 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:31.897134    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podStartSLOduration=10.89712102 podStartE2EDuration="10.89712102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:14:31.896090669 +0000 UTC m=+1769.276540109" watchObservedRunningTime="2026-10-30 11:14:31.89712102 +0000 UTC m=+1769.277570459"
Oct 30 11:14:39.429913 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:14:39.429874    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:14:43.816522 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:43.816482    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:14:43.817139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:43.817113    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:14:52.861981 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:14:52.861931    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:15:02.859492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:02.859450    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:15:02.950235 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:02.950140    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:15:02.955604 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:02.955585    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:15:09.432201 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:09.432163    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:15:12.860764 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:12.860726    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:15:29.595775 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.595739    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:29.596351 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.595982    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="d75010c3-33e0-4ce1-9a76-7fad7acda61f" containerName="otc-container" containerID="cri-o://9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab" gracePeriod=30
Oct 30 11:15:29.857060 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.857003    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:29.952089 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.952060    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d75010c3-33e0-4ce1-9a76-7fad7acda61f-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d75010c3-33e0-4ce1-9a76-7fad7acda61f-monitoring-tls-certs\") pod \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") "
Oct 30 11:15:29.952219 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.952110    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d75010c3-33e0-4ce1-9a76-7fad7acda61f-otc-internal\" (UniqueName: \"kubernetes.io/configmap/d75010c3-33e0-4ce1-9a76-7fad7acda61f-otc-internal\") pod \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") "
Oct 30 11:15:29.952219 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.952141    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d75010c3-33e0-4ce1-9a76-7fad7acda61f-kube-api-access-wkcxp\" (UniqueName: \"kubernetes.io/projected/d75010c3-33e0-4ce1-9a76-7fad7acda61f-kube-api-access-wkcxp\") pod \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\" (UID: \"d75010c3-33e0-4ce1-9a76-7fad7acda61f\") "
Oct 30 11:15:29.952496 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.952471    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d75010c3-33e0-4ce1-9a76-7fad7acda61f-otc-internal" pod "d75010c3-33e0-4ce1-9a76-7fad7acda61f" (UID: "d75010c3-33e0-4ce1-9a76-7fad7acda61f"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:15:29.954254 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.954226    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d75010c3-33e0-4ce1-9a76-7fad7acda61f-monitoring-tls-certs" pod "d75010c3-33e0-4ce1-9a76-7fad7acda61f" (UID: "d75010c3-33e0-4ce1-9a76-7fad7acda61f"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:15:29.954356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:29.954313    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d75010c3-33e0-4ce1-9a76-7fad7acda61f-kube-api-access-wkcxp" pod "d75010c3-33e0-4ce1-9a76-7fad7acda61f" (UID: "d75010c3-33e0-4ce1-9a76-7fad7acda61f"). InnerVolumeSpecName "kube-api-access-wkcxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:15:30.052964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.052942    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d75010c3-33e0-4ce1-9a76-7fad7acda61f-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:30.052964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.052963    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d75010c3-33e0-4ce1-9a76-7fad7acda61f-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:30.053120 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.052973    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkcxp\" (UniqueName: \"kubernetes.io/projected/d75010c3-33e0-4ce1-9a76-7fad7acda61f-kube-api-access-wkcxp\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:30.147658 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.147601    4849 generic.go:356] "Generic (PLEG): container finished" podID="d75010c3-33e0-4ce1-9a76-7fad7acda61f" containerID="9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab" exitCode=0
Oct 30 11:15:30.147658 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.147644    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.147780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.147685    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d75010c3-33e0-4ce1-9a76-7fad7acda61f","Type":"ContainerDied","Data":"9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab"}
Oct 30 11:15:30.147780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.147732    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d75010c3-33e0-4ce1-9a76-7fad7acda61f","Type":"ContainerDied","Data":"6d897a5084d72e5a48608e563fa4f1aed16aeeea89148ae2284e467e94db982c"}
Oct 30 11:15:30.147780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.147756    4849 scope.go:122] "RemoveContainer" containerID="9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab"
Oct 30 11:15:30.156972 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.156948    4849 scope.go:122] "RemoveContainer" containerID="9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab"
Oct 30 11:15:30.157311 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:30.157293    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab\": container with ID starting with 9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab not found: ID does not exist" containerID="9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab"
Oct 30 11:15:30.157385 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.157324    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab"} err="failed to get container status \"9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab\": rpc error: code = NotFound desc = could not find container \"9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab\": container with ID starting with 9e19bc81d063289f0c7e85cfe0fc3ffc84e610784f7706ff6af8772a076022ab not found: ID does not exist"
Oct 30 11:15:30.206049 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.206005    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:30.219475 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.219455    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:30.248757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.248719    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:30.249167 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.249147    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d75010c3-33e0-4ce1-9a76-7fad7acda61f" containerName="otc-container"
Oct 30 11:15:30.249167 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.249165    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d75010c3-33e0-4ce1-9a76-7fad7acda61f" containerName="otc-container"
Oct 30 11:15:30.249296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.249230    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d75010c3-33e0-4ce1-9a76-7fad7acda61f" containerName="otc-container"
Oct 30 11:15:30.253511 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.253495    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.269677 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.269657    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-29k2n\""
Oct 30 11:15:30.275454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.275436    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:30.275844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.275814    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-0ed87044\""
Oct 30 11:15:30.354042 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.354002    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcqtl\" (UniqueName: \"kubernetes.io/projected/7bcfcd2d-163c-4d9c-8038-b00b993618f9-kube-api-access-bcqtl\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.354147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.354059    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.354202 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.354168    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.370278 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.370257    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:15:30.454747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.454695    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.454747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.454727    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.454862 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.454791    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcqtl\" (UniqueName: \"kubernetes.io/projected/7bcfcd2d-163c-4d9c-8038-b00b993618f9-kube-api-access-bcqtl\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.455416 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.455386    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.457215 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.457197    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.479178 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.479156    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcqtl\" (UniqueName: \"kubernetes.io/projected/7bcfcd2d-163c-4d9c-8038-b00b993618f9-kube-api-access-bcqtl\") pod \"data-science-collector-collector-0\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.565901 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.565882    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:30.753782 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.753705    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d75010c3-33e0-4ce1-9a76-7fad7acda61f" path="/var/lib/kubelet/pods/d75010c3-33e0-4ce1-9a76-7fad7acda61f/volumes"
Oct 30 11:15:30.874977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.874956    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:30.877288 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:15:30.877266    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bcfcd2d_163c_4d9c_8038_b00b993618f9.slice/crio-e43101ed30686fdfb90cfd5d5c3c6f554d8804d03f72cb1a08da82bbf42f209e WatchSource:0}: Error finding container e43101ed30686fdfb90cfd5d5c3c6f554d8804d03f72cb1a08da82bbf42f209e: Status 404 returned error can't find the container with id e43101ed30686fdfb90cfd5d5c3c6f554d8804d03f72cb1a08da82bbf42f209e
Oct 30 11:15:30.879424 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.879392    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:15:30.879538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:30.879457    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:15:31.155399 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:31.155368    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"7bcfcd2d-163c-4d9c-8038-b00b993618f9","Type":"ContainerStarted","Data":"7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0"}
Oct 30 11:15:31.155568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:31.155428    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"7bcfcd2d-163c-4d9c-8038-b00b993618f9","Type":"ContainerStarted","Data":"e43101ed30686fdfb90cfd5d5c3c6f554d8804d03f72cb1a08da82bbf42f209e"}
Oct 30 11:15:31.203670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:31.203619    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.203595343 podStartE2EDuration="1.203595343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:15:31.200386405 +0000 UTC m=+1828.580835845" watchObservedRunningTime="2026-10-30 11:15:31.203595343 +0000 UTC m=+1828.584044785"
Oct 30 11:15:31.695484 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:31.695446    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:32.162752 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.162716    4849 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="redhat-ods-monitoring/data-science-collector-collector-0" secret="" err="secret \"data-science-collector-collector-dockercfg-29k2n\" not found"
Oct 30 11:15:32.270283 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.270241    4849 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-collector-collector-0ed87044: configmap "data-science-collector-collector-0ed87044" not found
Oct 30 11:15:32.270475 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.270358    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal podName:7bcfcd2d-163c-4d9c-8038-b00b993618f9 nodeName:}" failed. No retries permitted until 2026-10-30 11:15:32.770321337 +0000 UTC m=+1830.150770776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "otc-internal" (UniqueName: "kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal") pod "data-science-collector-collector-0" (UID: "7bcfcd2d-163c-4d9c-8038-b00b993618f9") : configmap "data-science-collector-collector-0ed87044" not found
Oct 30 11:15:32.270557 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.270520    4849 secret.go:189] Couldn't get secret redhat-ods-monitoring/data-science-collector-monitoring-tls: secret "data-science-collector-monitoring-tls" not found
Oct 30 11:15:32.271580 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.270664    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs podName:7bcfcd2d-163c-4d9c-8038-b00b993618f9 nodeName:}" failed. No retries permitted until 2026-10-30 11:15:32.77059646 +0000 UTC m=+1830.151045891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-tls-certs" (UniqueName: "kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs") pod "data-science-collector-collector-0" (UID: "7bcfcd2d-163c-4d9c-8038-b00b993618f9") : secret "data-science-collector-monitoring-tls" not found
Oct 30 11:15:32.274382 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.274337    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:15:32.274604 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.274555    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo" containerID="cri-o://f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163" gracePeriod=30
Oct 30 11:15:32.295094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.295068    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"]
Oct 30 11:15:32.295269 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.295250    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerName="tempo" containerID="cri-o://8671bc17a4b49dac9f4bc1cbdbdae4927753d982b7f0f7dc7e8f14634feb518d" gracePeriod=30
Oct 30 11:15:32.297988 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.297954    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"]
Oct 30 11:15:32.298377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.298344    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx" podUID="0c4a2809-031b-4aec-a51b-9c10d17c91a2" containerName="tempo" containerID="cri-o://be588b0395f9a0187c3199c36d584f1c77d17a137c0fcf1b9fd3315e82dad20c" gracePeriod=30
Oct 30 11:15:32.775394 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.775364    4849 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-collector-collector-0ed87044: configmap "data-science-collector-collector-0ed87044" not found
Oct 30 11:15:32.775554 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.775399    4849 secret.go:189] Couldn't get secret redhat-ods-monitoring/data-science-collector-monitoring-tls: secret "data-science-collector-monitoring-tls" not found
Oct 30 11:15:32.775554 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.775431    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal podName:7bcfcd2d-163c-4d9c-8038-b00b993618f9 nodeName:}" failed. No retries permitted until 2026-10-30 11:15:33.775419297 +0000 UTC m=+1831.155868714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "otc-internal" (UniqueName: "kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal") pod "data-science-collector-collector-0" (UID: "7bcfcd2d-163c-4d9c-8038-b00b993618f9") : configmap "data-science-collector-collector-0ed87044" not found
Oct 30 11:15:32.775554 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:32.775448    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs podName:7bcfcd2d-163c-4d9c-8038-b00b993618f9 nodeName:}" failed. No retries permitted until 2026-10-30 11:15:33.775436569 +0000 UTC m=+1831.155886011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-tls-certs" (UniqueName: "kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs") pod "data-science-collector-collector-0" (UID: "7bcfcd2d-163c-4d9c-8038-b00b993618f9") : secret "data-science-collector-monitoring-tls" not found
Oct 30 11:15:32.859967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:32.859920    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:15:33.173294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.172964    4849 generic.go:356] "Generic (PLEG): container finished" podID="0c4a2809-031b-4aec-a51b-9c10d17c91a2" containerID="be588b0395f9a0187c3199c36d584f1c77d17a137c0fcf1b9fd3315e82dad20c" exitCode=0
Oct 30 11:15:33.173294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.173078    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx" event={"ID":"0c4a2809-031b-4aec-a51b-9c10d17c91a2","Type":"ContainerDied","Data":"be588b0395f9a0187c3199c36d584f1c77d17a137c0fcf1b9fd3315e82dad20c"}
Oct 30 11:15:33.173835 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.173450    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="7bcfcd2d-163c-4d9c-8038-b00b993618f9" containerName="otc-container" containerID="cri-o://7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0" gracePeriod=30
Oct 30 11:15:33.302442 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.302417    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:15:33.379676 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.379649    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-distributor-mtls\") pod \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") "
Oct 30 11:15:33.379825 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.379717    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-tmp-storage\") pod \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") "
Oct 30 11:15:33.379877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.379823    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c4a2809-031b-4aec-a51b-9c10d17c91a2-kube-api-access-7h4mg\" (UniqueName: \"kubernetes.io/projected/0c4a2809-031b-4aec-a51b-9c10d17c91a2-kube-api-access-7h4mg\") pod \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") "
Oct 30 11:15:33.379877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.379861    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-ca-bundle\") pod \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") "
Oct 30 11:15:33.379989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.379898    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-conf\") pod \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\" (UID: \"0c4a2809-031b-4aec-a51b-9c10d17c91a2\") "
Oct 30 11:15:33.380218 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.380184    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-tmp-storage" pod "0c4a2809-031b-4aec-a51b-9c10d17c91a2" (UID: "0c4a2809-031b-4aec-a51b-9c10d17c91a2"). InnerVolumeSpecName "tempo-tmp-storage". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:15:33.380360 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.380228    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-ca-bundle" pod "0c4a2809-031b-4aec-a51b-9c10d17c91a2" (UID: "0c4a2809-031b-4aec-a51b-9c10d17c91a2"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:15:33.380415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.380388    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-conf" pod "0c4a2809-031b-4aec-a51b-9c10d17c91a2" (UID: "0c4a2809-031b-4aec-a51b-9c10d17c91a2"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:15:33.381897 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.381864    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-distributor-mtls" pod "0c4a2809-031b-4aec-a51b-9c10d17c91a2" (UID: "0c4a2809-031b-4aec-a51b-9c10d17c91a2"). InnerVolumeSpecName "tempo-data-science-tempostack-distributor-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:15:33.381988 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.381916    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4a2809-031b-4aec-a51b-9c10d17c91a2-kube-api-access-7h4mg" pod "0c4a2809-031b-4aec-a51b-9c10d17c91a2" (UID: "0c4a2809-031b-4aec-a51b-9c10d17c91a2"). InnerVolumeSpecName "kube-api-access-7h4mg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:15:33.425510 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.425489    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:33.481209 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481189    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs\") pod \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") "
Oct 30 11:15:33.481316 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481232    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal\" (UniqueName: \"kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal\") pod \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") "
Oct 30 11:15:33.481316 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481260    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bcfcd2d-163c-4d9c-8038-b00b993618f9-kube-api-access-bcqtl\" (UniqueName: \"kubernetes.io/projected/7bcfcd2d-163c-4d9c-8038-b00b993618f9-kube-api-access-bcqtl\") pod \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\" (UID: \"7bcfcd2d-163c-4d9c-8038-b00b993618f9\") "
Oct 30 11:15:33.481524 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481505    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h4mg\" (UniqueName: \"kubernetes.io/projected/0c4a2809-031b-4aec-a51b-9c10d17c91a2-kube-api-access-7h4mg\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.481598 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481532    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.481598 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481549    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.481598 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481551    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal" pod "7bcfcd2d-163c-4d9c-8038-b00b993618f9" (UID: "7bcfcd2d-163c-4d9c-8038-b00b993618f9"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:15:33.481598 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481563    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-data-science-tempostack-distributor-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.481598 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.481578    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0c4a2809-031b-4aec-a51b-9c10d17c91a2-tempo-tmp-storage\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.483094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.483072    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs" pod "7bcfcd2d-163c-4d9c-8038-b00b993618f9" (UID: "7bcfcd2d-163c-4d9c-8038-b00b993618f9"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:15:33.483251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.483233    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bcfcd2d-163c-4d9c-8038-b00b993618f9-kube-api-access-bcqtl" pod "7bcfcd2d-163c-4d9c-8038-b00b993618f9" (UID: "7bcfcd2d-163c-4d9c-8038-b00b993618f9"). InnerVolumeSpecName "kube-api-access-bcqtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:15:33.582859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.582836    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcqtl\" (UniqueName: \"kubernetes.io/projected/7bcfcd2d-163c-4d9c-8038-b00b993618f9-kube-api-access-bcqtl\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.582859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.582855    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/7bcfcd2d-163c-4d9c-8038-b00b993618f9-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.582987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.582865    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/7bcfcd2d-163c-4d9c-8038-b00b993618f9-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:33.814658 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:33.814626    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:15:34.179376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.179350    4849 generic.go:356] "Generic (PLEG): container finished" podID="7bcfcd2d-163c-4d9c-8038-b00b993618f9" containerID="7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0" exitCode=0
Oct 30 11:15:34.179697 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.179402    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:15:34.179697 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.179426    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"7bcfcd2d-163c-4d9c-8038-b00b993618f9","Type":"ContainerDied","Data":"7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0"}
Oct 30 11:15:34.179697 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.179481    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"7bcfcd2d-163c-4d9c-8038-b00b993618f9","Type":"ContainerDied","Data":"e43101ed30686fdfb90cfd5d5c3c6f554d8804d03f72cb1a08da82bbf42f209e"}
Oct 30 11:15:34.179697 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.179511    4849 scope.go:122] "RemoveContainer" containerID="7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0"
Oct 30 11:15:34.181157 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.181135    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"
Oct 30 11:15:34.181295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.181156    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx" event={"ID":"0c4a2809-031b-4aec-a51b-9c10d17c91a2","Type":"ContainerDied","Data":"e62acde65b9fc1bfff483097b6fd748700f0519ecb4830c5e594aaec0cfa6c24"}
Oct 30 11:15:34.189204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.189185    4849 scope.go:122] "RemoveContainer" containerID="7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0"
Oct 30 11:15:34.189539 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:34.189515    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0\": container with ID starting with 7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0 not found: ID does not exist" containerID="7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0"
Oct 30 11:15:34.189649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.189551    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0"} err="failed to get container status \"7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0\": rpc error: code = NotFound desc = could not find container \"7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0\": container with ID starting with 7a1cd55f4c2fb73c8bd4978d5a26d6f3d03afabf25609b18fde95c4464388ba0 not found: ID does not exist"
Oct 30 11:15:34.189649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.189576    4849 scope.go:122] "RemoveContainer" containerID="be588b0395f9a0187c3199c36d584f1c77d17a137c0fcf1b9fd3315e82dad20c"
Oct 30 11:15:34.235654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.235631    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"]
Oct 30 11:15:34.249059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.249038    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-8677bdc7b8-zp7fx"]
Oct 30 11:15:34.285152 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.285131    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:34.289207 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.289187    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:15:34.753387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.753362    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c4a2809-031b-4aec-a51b-9c10d17c91a2" path="/var/lib/kubelet/pods/0c4a2809-031b-4aec-a51b-9c10d17c91a2/volumes"
Oct 30 11:15:34.753788 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:34.753772    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bcfcd2d-163c-4d9c-8038-b00b993618f9" path="/var/lib/kubelet/pods/7bcfcd2d-163c-4d9c-8038-b00b993618f9/volumes"
Oct 30 11:15:39.430666 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:39.430598    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:15:42.860135 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:42.860092    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:15:43.814610 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:43.814573    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:15:52.856298 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:52.856264    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo" probeResult="failure" output="Get \"https://10.132.0.76:3101/ready\": dial tcp 10.132.0.76:3101: connect: connection refused"
Oct 30 11:15:52.856611 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:52.856359    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:15:53.020205 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.020180    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:15:53.133461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.133438    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") pod \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") "
Oct 30 11:15:53.133625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.133534    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ca-bundle\") pod \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") "
Oct 30 11:15:53.133625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.133588    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-conf\") pod \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") "
Oct 30 11:15:53.133754 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.133708    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ingester-mtls\") pod \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") "
Oct 30 11:15:53.133820 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.133766    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-kube-api-access-btj9h\" (UniqueName: \"kubernetes.io/projected/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-kube-api-access-btj9h\") pod \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\" (UID: \"b880b417-ad1b-4c9f-85ad-6946b3ff76f9\") "
Oct 30 11:15:53.134041 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.133937    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ca-bundle" pod "b880b417-ad1b-4c9f-85ad-6946b3ff76f9" (UID: "b880b417-ad1b-4c9f-85ad-6946b3ff76f9"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:15:53.134041 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.133979    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-conf" pod "b880b417-ad1b-4c9f-85ad-6946b3ff76f9" (UID: "b880b417-ad1b-4c9f-85ad-6946b3ff76f9"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:15:53.134224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.134082    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:53.134224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.134099    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:53.135900 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.135876    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ingester-mtls" pod "b880b417-ad1b-4c9f-85ad-6946b3ff76f9" (UID: "b880b417-ad1b-4c9f-85ad-6946b3ff76f9"). InnerVolumeSpecName "tempo-data-science-tempostack-ingester-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:15:53.136061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.135960    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-kube-api-access-btj9h" pod "b880b417-ad1b-4c9f-85ad-6946b3ff76f9" (UID: "b880b417-ad1b-4c9f-85ad-6946b3ff76f9"). InnerVolumeSpecName "kube-api-access-btj9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:15:53.136061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.136050    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4" pod "b880b417-ad1b-4c9f-85ad-6946b3ff76f9" (UID: "b880b417-ad1b-4c9f-85ad-6946b3ff76f9"). InnerVolumeSpecName "pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:15:53.234790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.234763    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:15:53.234790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.234787    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-tempo-data-science-tempostack-ingester-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:53.234970 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.234802    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btj9h\" (UniqueName: \"kubernetes.io/projected/b880b417-ad1b-4c9f-85ad-6946b3ff76f9-kube-api-access-btj9h\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:15:53.244502 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.244487    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:15:53.281673 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.281648    4849 generic.go:356] "Generic (PLEG): container finished" podID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerID="f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163" exitCode=0
Oct 30 11:15:53.281768 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.281690    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" event={"ID":"b880b417-ad1b-4c9f-85ad-6946b3ff76f9","Type":"ContainerDied","Data":"f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163"}
Oct 30 11:15:53.281768 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.281696    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:15:53.281768 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.281729    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" event={"ID":"b880b417-ad1b-4c9f-85ad-6946b3ff76f9","Type":"ContainerDied","Data":"c1f9f8cd840420f84f39b5eadf3198413f271bd6f66004b76c2fa4b058ba8a14"}
Oct 30 11:15:53.281768 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.281751    4849 scope.go:122] "RemoveContainer" containerID="f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163"
Oct 30 11:15:53.290894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.290874    4849 scope.go:122] "RemoveContainer" containerID="f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163"
Oct 30 11:15:53.291171 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:15:53.291153    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163\": container with ID starting with f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163 not found: ID does not exist" containerID="f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163"
Oct 30 11:15:53.291245 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.291179    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163"} err="failed to get container status \"f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163\": rpc error: code = NotFound desc = could not find container \"f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163\": container with ID starting with f497287cad55f05ae5ae1ad5ece7c3ec7e9ee796989c4f1468d78a73b733e163 not found: ID does not exist"
Oct 30 11:15:53.335759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.335736    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-aedcdca3-95dc-4504-a60a-a57d130b1e02\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c4fd50f939667bd4\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-7f600dab7f8bdfb34606269590de3813f736c88298ef584d5dfaa352a938aaad\""
Oct 30 11:15:53.337643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.337622    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:15:53.351238 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.351213    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:15:53.814370 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.814344    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:15:53.814499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:53.814459    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:15:54.753208 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:15:54.753177    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" path="/var/lib/kubelet/pods/b880b417-ad1b-4c9f-85ad-6946b3ff76f9/volumes"
Oct 30 11:16:02.330873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.330840    4849 generic.go:356] "Generic (PLEG): container finished" podID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerID="8671bc17a4b49dac9f4bc1cbdbdae4927753d982b7f0f7dc7e8f14634feb518d" exitCode=137
Oct 30 11:16:02.331243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.330927    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" event={"ID":"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7","Type":"ContainerDied","Data":"8671bc17a4b49dac9f4bc1cbdbdae4927753d982b7f0f7dc7e8f14634feb518d"}
Oct 30 11:16:02.446985 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.446957    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:16:02.602813 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.602784    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-querier-mtls\") pod \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") "
Oct 30 11:16:02.602969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.602828    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-kube-api-access-5pfxw\" (UniqueName: \"kubernetes.io/projected/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-kube-api-access-5pfxw\") pod \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") "
Oct 30 11:16:02.602969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.602902    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-tmp-storage\") pod \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") "
Oct 30 11:16:02.602969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.602924    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-ca-bundle\") pod \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") "
Oct 30 11:16:02.602969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.602952    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-conf\") pod \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\" (UID: \"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7\") "
Oct 30 11:16:02.603345 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.603303    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-tmp-storage" pod "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" (UID: "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7"). InnerVolumeSpecName "tempo-tmp-storage". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:16:02.603345 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.603329    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-ca-bundle" pod "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" (UID: "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:16:02.603498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.603359    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-conf" pod "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" (UID: "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:16:02.604975 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.604955    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-kube-api-access-5pfxw" pod "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" (UID: "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7"). InnerVolumeSpecName "kube-api-access-5pfxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:16:02.605079 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.604976    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-querier-mtls" pod "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" (UID: "0bbc89f0-d2e0-43cc-baa5-ba88df275fd7"). InnerVolumeSpecName "tempo-data-science-tempostack-querier-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:16:02.704446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.704409    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-querier-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:02.704446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.704440    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pfxw\" (UniqueName: \"kubernetes.io/projected/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-kube-api-access-5pfxw\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:02.704654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.704454    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-tmp-storage\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:02.704654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.704468    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:02.704654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:02.704484    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:03.336415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:03.336385    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf" event={"ID":"0bbc89f0-d2e0-43cc-baa5-ba88df275fd7","Type":"ContainerDied","Data":"c9f9a6419a844e6ed34a2d35a454223825749231045f9ca8d8ceb1c7791ef180"}
Oct 30 11:16:03.336775 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:03.336433    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"
Oct 30 11:16:03.336775 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:03.336467    4849 scope.go:122] "RemoveContainer" containerID="8671bc17a4b49dac9f4bc1cbdbdae4927753d982b7f0f7dc7e8f14634feb518d"
Oct 30 11:16:03.392829 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:03.392773    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"]
Oct 30 11:16:03.406247 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:03.406224    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-59c88557df-tc7qf"]
Oct 30 11:16:04.754237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:04.754205    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" path="/var/lib/kubelet/pods/0bbc89f0-d2e0-43cc-baa5-ba88df275fd7/volumes"
Oct 30 11:16:09.432628 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:16:09.432589    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:16:18.554641 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554606    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554936    4849 cpu_manager.go:414] "containerMap: removing container" podUID="0c4a2809-031b-4aec-a51b-9c10d17c91a2" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554949    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c4a2809-031b-4aec-a51b-9c10d17c91a2" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554962    4849 cpu_manager.go:414] "containerMap: removing container" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554966    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554978    4849 cpu_manager.go:414] "containerMap: removing container" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554983    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.554999    4849 cpu_manager.go:414] "containerMap: removing container" podUID="7bcfcd2d-163c-4d9c-8038-b00b993618f9" containerName="otc-container"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.555004    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bcfcd2d-163c-4d9c-8038-b00b993618f9" containerName="otc-container"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.555112    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c4a2809-031b-4aec-a51b-9c10d17c91a2" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.555121    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="b880b417-ad1b-4c9f-85ad-6946b3ff76f9" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.555128    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bbc89f0-d2e0-43cc-baa5-ba88df275fd7" containerName="tempo"
Oct 30 11:16:18.555183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.555136    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bcfcd2d-163c-4d9c-8038-b00b993618f9" containerName="otc-container"
Oct 30 11:16:18.559847 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.559825    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.580223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.580199    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-0ed87044\""
Oct 30 11:16:18.593561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.593540    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-789dw\""
Oct 30 11:16:18.593672 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.593582    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:16:18.595969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.595945    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:16:18.682633 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.682606    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:16:18.682789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.682762    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:16:18.722337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.722307    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.722436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.722351    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmlkz\" (UniqueName: \"kubernetes.io/projected/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-kube-api-access-hmlkz\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.722478 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.722445    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.823383 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.823356    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.823495 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.823387    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmlkz\" (UniqueName: \"kubernetes.io/projected/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-kube-api-access-hmlkz\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.823495 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.823418    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.823967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.823950    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.825861 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.825842    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.848052 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.848006    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmlkz\" (UniqueName: \"kubernetes.io/projected/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-kube-api-access-hmlkz\") pod \"data-science-collector-collector-0\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:18.873250 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:18.873232    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:16:19.176271 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:19.176250    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:16:19.178296 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:16:19.178267    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0ee2d4_0884_4dd2_b72a_897494c1b12a.slice/crio-5c271cb1f2f88905925fbfc30672fa35b30178ec49f004e6a03907bd3f726be4 WatchSource:0}: Error finding container 5c271cb1f2f88905925fbfc30672fa35b30178ec49f004e6a03907bd3f726be4: Status 404 returned error can't find the container with id 5c271cb1f2f88905925fbfc30672fa35b30178ec49f004e6a03907bd3f726be4
Oct 30 11:16:19.180281 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:19.180249    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:16:19.180397 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:19.180315    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:16:19.417162 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:19.417125    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ec0ee2d4-0884-4dd2-b72a-897494c1b12a","Type":"ContainerStarted","Data":"8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11"}
Oct 30 11:16:19.417321 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:19.417189    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ec0ee2d4-0884-4dd2-b72a-897494c1b12a","Type":"ContainerStarted","Data":"5c271cb1f2f88905925fbfc30672fa35b30178ec49f004e6a03907bd3f726be4"}
Oct 30 11:16:19.462142 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:19.462065    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.462052325 podStartE2EDuration="1.462052325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:16:19.460443597 +0000 UTC m=+1876.840893036" watchObservedRunningTime="2026-10-30 11:16:19.462052325 +0000 UTC m=+1876.842501764"
Oct 30 11:16:22.692407 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.692376    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"]
Oct 30 11:16:22.695857 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.695841    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.714362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.714342    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-distributor-mtls\""
Oct 30 11:16:22.714469 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.714428    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-dockercfg-264xv\""
Oct 30 11:16:22.727668 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.727645    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-ca-bundle\""
Oct 30 11:16:22.728219 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.728201    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack\""
Oct 30 11:16:22.730077 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.730057    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"]
Oct 30 11:16:22.746466 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.746438    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:16:22.749898 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.749881    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.784993 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.784968    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:16:22.813561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.813541    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"gcs-secret\""
Oct 30 11:16:22.813703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.813673    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-ingester-mtls\""
Oct 30 11:16:22.852586 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852562    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd5p4\" (UniqueName: \"kubernetes.io/projected/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-kube-api-access-nd5p4\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.852699 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852594    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-distributor-mtls\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.852699 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852612    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt54s\" (UniqueName: \"kubernetes.io/projected/b0275257-4f71-4017-ab70-bd04dac34f41-kube-api-access-mt54s\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.852699 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852667    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.852904 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852704    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-conf\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.852904 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852828    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-storage-gcs-key\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.853013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852904    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ingester-mtls\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.853013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.852941    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.853013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.853003    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.853159 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.853108    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-conf\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.904764 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.904737    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"]
Oct 30 11:16:22.908751 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.908730    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:22.932905 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.932879    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"]
Oct 30 11:16:22.953834 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.953775    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.953834 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.953805    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-conf\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.953964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.953834    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:22.953964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.953877    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-querier-mtls\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:22.953964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.953898    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:22.953964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.953922    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-storage-gcs-key\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.954147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.953979    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ingester-mtls\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.954147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954003    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-conf\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:22.954147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954044    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.954147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954072    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.954147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954092    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-storage-gcs-key\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:22.954147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954109    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfqn4\" (UniqueName: \"kubernetes.io/projected/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-kube-api-access-xfqn4\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:22.954147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954131    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-conf\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.954376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954179    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd5p4\" (UniqueName: \"kubernetes.io/projected/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-kube-api-access-nd5p4\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.954376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954197    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-distributor-mtls\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.954376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.954213    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt54s\" (UniqueName: \"kubernetes.io/projected/b0275257-4f71-4017-ab70-bd04dac34f41-kube-api-access-mt54s\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.956823 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.956803    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.957416 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.957398    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-conf\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.958588 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.958317    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.958588 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.958544    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.958753 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.958682    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-conf\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.960268 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.960249    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ingester-mtls\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.960632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.960608    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-distributor-mtls\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:22.961112 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.961090    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-storage-gcs-key\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.966615 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.966597    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-querier-mtls\""
Oct 30 11:16:22.978294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.978275    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt54s\" (UniqueName: \"kubernetes.io/projected/b0275257-4f71-4017-ab70-bd04dac34f41-kube-api-access-mt54s\") pod \"tempo-data-science-tempostack-ingester-0\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:22.986947 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:22.986925    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd5p4\" (UniqueName: \"kubernetes.io/projected/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-kube-api-access-nd5p4\") pod \"tempo-data-science-tempostack-distributor-544f96d8b5-r97nr\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:23.009331 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.009313    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:23.055275 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.055247    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-conf\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.055425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.055293    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-storage-gcs-key\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.055425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.055308    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfqn4\" (UniqueName: \"kubernetes.io/projected/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-kube-api-access-xfqn4\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.055425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.055366    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.055425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.055392    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-querier-mtls\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.055635 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.055606    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.056004 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.055956    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.056331 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.056312    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.056630 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.056604    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-conf\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.058083 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.058057    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-querier-mtls\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.058196 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.058086    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-storage-gcs-key\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.080104 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.080082    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfqn4\" (UniqueName: \"kubernetes.io/projected/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-kube-api-access-xfqn4\") pod \"tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.224957 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.224902    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:23.330359 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.330330    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"]
Oct 30 11:16:23.334650 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:16:23.334624    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef28dccc_28f1_4ab7_96ac_b7cf0ca4f902.slice/crio-2850b39475833253df6bd0c38132b922fed3d199b1045517df1d607549af5e7f WatchSource:0}: Error finding container 2850b39475833253df6bd0c38132b922fed3d199b1045517df1d607549af5e7f: Status 404 returned error can't find the container with id 2850b39475833253df6bd0c38132b922fed3d199b1045517df1d607549af5e7f
Oct 30 11:16:23.437458 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.437427    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr" event={"ID":"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902","Type":"ContainerStarted","Data":"bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5"}
Oct 30 11:16:23.437558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.437486    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr" event={"ID":"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902","Type":"ContainerStarted","Data":"2850b39475833253df6bd0c38132b922fed3d199b1045517df1d607549af5e7f"}
Oct 30 11:16:23.437558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.437527    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:23.483994 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.483911    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr" podStartSLOduration=1.483894993 podStartE2EDuration="1.483894993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:16:23.481960572 +0000 UTC m=+1880.862410013" watchObservedRunningTime="2026-10-30 11:16:23.483894993 +0000 UTC m=+1880.864344433"
Oct 30 11:16:23.509314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.509289    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:16:23.526296 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:16:23.526268    4849 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[data], unattached volumes=[data], failed to process volumes=[]: context canceled" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0" podUID="b0275257-4f71-4017-ab70-bd04dac34f41"
Oct 30 11:16:23.533538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.533501    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"]
Oct 30 11:16:23.535760 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:16:23.535732    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b66db9a_4de7_4eac_aee6_2c289b22c3ed.slice/crio-41161bb280ec8b303227fb873b4b3301d40e3dbdaa20f4b75ef1fba07ad6b35a WatchSource:0}: Error finding container 41161bb280ec8b303227fb873b4b3301d40e3dbdaa20f4b75ef1fba07ad6b35a: Status 404 returned error can't find the container with id 41161bb280ec8b303227fb873b4b3301d40e3dbdaa20f4b75ef1fba07ad6b35a
Oct 30 11:16:23.590460 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.590435    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"]
Oct 30 11:16:23.593955 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.593939    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.617882 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.617859    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"]
Oct 30 11:16:23.637298 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.637275    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-query-frontend-mtls\""
Oct 30 11:16:23.643101 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.643066    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"]
Oct 30 11:16:23.647004 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.646982    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.661429 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.661392    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-query-frontend-mtls\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-query-frontend-mtls\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.661544 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.661518    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-storage-gcs-key\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.661619 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.661543    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n96m7\" (UniqueName: \"kubernetes.io/projected/381e80c9-06cb-4461-b710-e068648ba579-kube-api-access-n96m7\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.661619 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.661591    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-conf\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.661729 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.661638    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/381e80c9-06cb-4461-b710-e068648ba579-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.661729 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.661671    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.680241 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.680217    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"]
Oct 30 11:16:23.737808 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.737755    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"tempo-data-science-tempostack-compactor-mtls\""
Oct 30 11:16:23.762591 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762563    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-storage-gcs-key\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.762703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762602    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96m7\" (UniqueName: \"kubernetes.io/projected/381e80c9-06cb-4461-b710-e068648ba579-kube-api-access-n96m7\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.762703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762636    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-conf\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.762703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762669    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-conf\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.762703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762697    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-storage-gcs-key\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.762921 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762726    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.762921 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762874    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-compactor-mtls\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-compactor-mtls\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.763046 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762925    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/381e80c9-06cb-4461-b710-e068648ba579-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.763046 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762957    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.763046 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.762987    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.763266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.763105    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkmvc\" (UniqueName: \"kubernetes.io/projected/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-kube-api-access-wkmvc\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.763266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.763149    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-query-frontend-mtls\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-query-frontend-mtls\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.763374 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.763316    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/381e80c9-06cb-4461-b710-e068648ba579-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.763626 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.763601    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.763789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.763766    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-conf\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.765128 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.765107    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-storage-gcs-key\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.765361 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.765344    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-query-frontend-mtls\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-query-frontend-mtls\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.786398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.786370    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n96m7\" (UniqueName: \"kubernetes.io/projected/381e80c9-06cb-4461-b710-e068648ba579-kube-api-access-n96m7\") pod \"tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.864501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.864477    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkmvc\" (UniqueName: \"kubernetes.io/projected/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-kube-api-access-wkmvc\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.864607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.864560    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-conf\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.864607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.864578    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-storage-gcs-key\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.864607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.864596    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.864790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.864642    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-compactor-mtls\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-compactor-mtls\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.864790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.864688    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.865053 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.865004    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-tmp-storage\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.865415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.865394    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-ca-bundle\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.865589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.865573    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-conf\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.866968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.866936    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-storage-gcs-key\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.867226 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.867206    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tempo-data-science-tempostack-compactor-mtls\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-compactor-mtls\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.891228 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.891208    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkmvc\" (UniqueName: \"kubernetes.io/projected/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-kube-api-access-wkmvc\") pod \"tempo-data-science-tempostack-compactor-69bcb95f96-6brjd\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") " pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:23.907058 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.907038    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:23.965057 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:23.960407    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:24.221610 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.221588    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"]
Oct 30 11:16:24.222865 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:16:24.222842    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381e80c9_06cb_4461_b710_e068648ba579.slice/crio-4699e79d8daf634472f482ebec6a24fe0d443690c15ae1132a744d165d8ffe38 WatchSource:0}: Error finding container 4699e79d8daf634472f482ebec6a24fe0d443690c15ae1132a744d165d8ffe38: Status 404 returned error can't find the container with id 4699e79d8daf634472f482ebec6a24fe0d443690c15ae1132a744d165d8ffe38
Oct 30 11:16:24.277056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.273837    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"]
Oct 30 11:16:24.443599 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.443572    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" event={"ID":"aecc096f-f6a7-4105-9345-c2c6c3c5fce1","Type":"ContainerStarted","Data":"82ee2fc7f4dbf575f9447694daa939fd02b2742c977081be315182b2b367fa08"}
Oct 30 11:16:24.443714 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.443633    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:16:24.443714 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.443656    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" event={"ID":"aecc096f-f6a7-4105-9345-c2c6c3c5fce1","Type":"ContainerStarted","Data":"84d2ce0a58fcaf2a090f44b812aacbe9de28a75344d2d37fa03a1dcee3ea5c7a"}
Oct 30 11:16:24.445199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.445172    4849 generic.go:356] "Generic (PLEG): container finished" podID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerID="97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913" exitCode=1
Oct 30 11:16:24.445293 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.445257    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" event={"ID":"9b66db9a-4de7-4eac-aee6-2c289b22c3ed","Type":"ContainerDied","Data":"97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913"}
Oct 30 11:16:24.445339 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.445299    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" event={"ID":"9b66db9a-4de7-4eac-aee6-2c289b22c3ed","Type":"ContainerStarted","Data":"41161bb280ec8b303227fb873b4b3301d40e3dbdaa20f4b75ef1fba07ad6b35a"}
Oct 30 11:16:24.445450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.445434    4849 scope.go:122] "RemoveContainer" containerID="97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913"
Oct 30 11:16:24.446605 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.446582    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" event={"ID":"381e80c9-06cb-4461-b710-e068648ba579","Type":"ContainerStarted","Data":"4699e79d8daf634472f482ebec6a24fe0d443690c15ae1132a744d165d8ffe38"}
Oct 30 11:16:24.446702 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.446615    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:24.454397 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.454189    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:24.491373 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.491331    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podStartSLOduration=1.491318026 podStartE2EDuration="1.491318026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:16:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:16:24.490256302 +0000 UTC m=+1881.870705742" watchObservedRunningTime="2026-10-30 11:16:24.491318026 +0000 UTC m=+1881.871767464"
Oct 30 11:16:24.573695 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.573665    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ingester-mtls\") pod \"b0275257-4f71-4017-ab70-bd04dac34f41\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") "
Oct 30 11:16:24.573817 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.573754    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-conf\") pod \"b0275257-4f71-4017-ab70-bd04dac34f41\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") "
Oct 30 11:16:24.573886 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.573814    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ca-bundle\") pod \"b0275257-4f71-4017-ab70-bd04dac34f41\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") "
Oct 30 11:16:24.573886 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.573850    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0275257-4f71-4017-ab70-bd04dac34f41-kube-api-access-mt54s\" (UniqueName: \"kubernetes.io/projected/b0275257-4f71-4017-ab70-bd04dac34f41-kube-api-access-mt54s\") pod \"b0275257-4f71-4017-ab70-bd04dac34f41\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") "
Oct 30 11:16:24.573984 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.573918    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-storage-gcs-key\") pod \"b0275257-4f71-4017-ab70-bd04dac34f41\" (UID: \"b0275257-4f71-4017-ab70-bd04dac34f41\") "
Oct 30 11:16:24.574759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.574253    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-conf" pod "b0275257-4f71-4017-ab70-bd04dac34f41" (UID: "b0275257-4f71-4017-ab70-bd04dac34f41"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:16:24.574759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.574514    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:24.575422 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.575358    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ca-bundle" pod "b0275257-4f71-4017-ab70-bd04dac34f41" (UID: "b0275257-4f71-4017-ab70-bd04dac34f41"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:16:24.577200 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.577179    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-storage-gcs-key" pod "b0275257-4f71-4017-ab70-bd04dac34f41" (UID: "b0275257-4f71-4017-ab70-bd04dac34f41"). InnerVolumeSpecName "storage-gcs-key". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:16:24.577300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.577274    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ingester-mtls" pod "b0275257-4f71-4017-ab70-bd04dac34f41" (UID: "b0275257-4f71-4017-ab70-bd04dac34f41"). InnerVolumeSpecName "tempo-data-science-tempostack-ingester-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:16:24.577655 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.577632    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0275257-4f71-4017-ab70-bd04dac34f41-kube-api-access-mt54s" pod "b0275257-4f71-4017-ab70-bd04dac34f41" (UID: "b0275257-4f71-4017-ab70-bd04dac34f41"). InnerVolumeSpecName "kube-api-access-mt54s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:16:24.675762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.675737    4849 reconciler_common.go:299] "Volume detached for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-storage-gcs-key\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:24.675762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.675768    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ingester-mtls\" (UniqueName: \"kubernetes.io/secret/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ingester-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:24.675904 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.675785    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0275257-4f71-4017-ab70-bd04dac34f41-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:24.675904 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:24.675800    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt54s\" (UniqueName: \"kubernetes.io/projected/b0275257-4f71-4017-ab70-bd04dac34f41-kube-api-access-mt54s\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:25.453948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.453916    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" event={"ID":"9b66db9a-4de7-4eac-aee6-2c289b22c3ed","Type":"ContainerStarted","Data":"254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933"}
Oct 30 11:16:25.454430 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.453977    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:16:25.455523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.455501    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"
Oct 30 11:16:25.455640 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.455493    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" event={"ID":"381e80c9-06cb-4461-b710-e068648ba579","Type":"ContainerStarted","Data":"8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2"}
Oct 30 11:16:25.501701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.501658    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" podStartSLOduration=3.5016462710000003 podStartE2EDuration="3.501646271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:16:25.49997825 +0000 UTC m=+1882.880427689" watchObservedRunningTime="2026-10-30 11:16:25.501646271 +0000 UTC m=+1882.882095711"
Oct 30 11:16:25.546354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.546313    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" podStartSLOduration=2.546301354 podStartE2EDuration="2.546301354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:16:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:16:25.544910768 +0000 UTC m=+1882.925360208" watchObservedRunningTime="2026-10-30 11:16:25.546301354 +0000 UTC m=+1882.926750792"
Oct 30 11:16:25.615334 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.615303    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:16:25.621074 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:25.621053    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-ingester-0"]
Oct 30 11:16:26.461033 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:26.461005    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:26.753571 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:26.753494    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0275257-4f71-4017-ab70-bd04dac34f41" path="/var/lib/kubelet/pods/b0275257-4f71-4017-ab70-bd04dac34f41/volumes"
Oct 30 11:16:39.431346 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:16:39.431312    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:16:44.452758 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:44.452726    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:45.460664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:45.460621    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:16:45.839503 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:45.839465    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"]
Oct 30 11:16:45.839784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:45.839735    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo" containerID="cri-o://254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933" gracePeriod=30
Oct 30 11:16:45.846570 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:45.846538    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:16:45.895086 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:45.895059    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"]
Oct 30 11:16:45.895230 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:45.895212    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr" podUID="ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" containerName="tempo" containerID="cri-o://bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5" gracePeriod=30
Oct 30 11:16:46.748512 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.748484    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:46.851046 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.850978    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-distributor-mtls\") pod \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") "
Oct 30 11:16:46.851157 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.851046    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-conf\") pod \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") "
Oct 30 11:16:46.851157 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.851088    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-tmp-storage\") pod \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") "
Oct 30 11:16:46.851157 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.851128    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-ca-bundle\") pod \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") "
Oct 30 11:16:46.851273 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.851185    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-kube-api-access-nd5p4\" (UniqueName: \"kubernetes.io/projected/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-kube-api-access-nd5p4\") pod \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\" (UID: \"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902\") "
Oct 30 11:16:46.851440 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.851374    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-tmp-storage" pod "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" (UID: "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902"). InnerVolumeSpecName "tempo-tmp-storage". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:16:46.851532 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.851514    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-conf" pod "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" (UID: "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:16:46.851582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.851524    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-ca-bundle" pod "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" (UID: "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:16:46.853266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.853243    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-kube-api-access-nd5p4" pod "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" (UID: "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902"). InnerVolumeSpecName "kube-api-access-nd5p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:16:46.853266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.853266    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-distributor-mtls" pod "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" (UID: "ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902"). InnerVolumeSpecName "tempo-data-science-tempostack-distributor-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:16:46.952831 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.952810    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-distributor-mtls\" (UniqueName: \"kubernetes.io/secret/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-distributor-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:46.952831 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.952833    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:46.952949 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.952842    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-tmp-storage\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:46.952949 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.952852    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:46.952949 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:46.952864    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd5p4\" (UniqueName: \"kubernetes.io/projected/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902-kube-api-access-nd5p4\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:16:47.575579 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.575543    4849 generic.go:356] "Generic (PLEG): container finished" podID="ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" containerID="bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5" exitCode=0
Oct 30 11:16:47.575731 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.575590    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr" event={"ID":"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902","Type":"ContainerDied","Data":"bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5"}
Oct 30 11:16:47.575731 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.575599    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"
Oct 30 11:16:47.575731 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.575648    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr" event={"ID":"ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902","Type":"ContainerDied","Data":"2850b39475833253df6bd0c38132b922fed3d199b1045517df1d607549af5e7f"}
Oct 30 11:16:47.575731 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.575680    4849 scope.go:122] "RemoveContainer" containerID="bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5"
Oct 30 11:16:47.586487 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.586469    4849 scope.go:122] "RemoveContainer" containerID="bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5"
Oct 30 11:16:47.586778 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:16:47.586753    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5\": container with ID starting with bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5 not found: ID does not exist" containerID="bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5"
Oct 30 11:16:47.586894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.586791    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5"} err="failed to get container status \"bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5\": rpc error: code = NotFound desc = could not find container \"bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5\": container with ID starting with bd244c758f237581cd2c984b8fe6d003d39e478e07756b03eaad65976f5e7aa5 not found: ID does not exist"
Oct 30 11:16:47.631362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.631333    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"]
Oct 30 11:16:47.644663 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.644638    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-distributor-544f96d8b5-r97nr"]
Oct 30 11:16:47.668869 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:47.668849    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:16:48.753660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:48.753622    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" path="/var/lib/kubelet/pods/ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902/volumes"
Oct 30 11:16:55.460179 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:55.460134    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:16:55.844548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:16:55.844510    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:17:05.459987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:05.459937    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:17:05.844186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:05.844148    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:17:09.431921 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:17:09.431885    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:17:15.460383 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:15.460290    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:17:15.845016 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:15.844974    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:17:15.885817 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:17:15.885763    4849 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b66db9a_4de7_4eac_aee6_2c289b22c3ed.slice/crio-254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b66db9a_4de7_4eac_aee6_2c289b22c3ed.slice/crio-conmon-254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 11:17:15.888411 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:17:15.888374    4849 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b66db9a_4de7_4eac_aee6_2c289b22c3ed.slice/crio-254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b66db9a_4de7_4eac_aee6_2c289b22c3ed.slice/crio-conmon-254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 11:17:16.005418 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.005388    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:17:16.072295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072212    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-tmp-storage\") pod \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") "
Oct 30 11:17:16.072426 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072382    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-kube-api-access-xfqn4\" (UniqueName: \"kubernetes.io/projected/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-kube-api-access-xfqn4\") pod \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") "
Oct 30 11:17:16.072488 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072427    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-querier-mtls\") pod \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") "
Oct 30 11:17:16.072488 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072459    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-conf\") pod \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") "
Oct 30 11:17:16.072603 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072486    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-tmp-storage" pod "9b66db9a-4de7-4eac-aee6-2c289b22c3ed" (UID: "9b66db9a-4de7-4eac-aee6-2c289b22c3ed"). InnerVolumeSpecName "tempo-tmp-storage". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:17:16.072603 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072552    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-storage-gcs-key\") pod \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") "
Oct 30 11:17:16.072603 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072588    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-ca-bundle\") pod \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\" (UID: \"9b66db9a-4de7-4eac-aee6-2c289b22c3ed\") "
Oct 30 11:17:16.072858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.072827    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-tmp-storage\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:16.073055 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.073012    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-conf" pod "9b66db9a-4de7-4eac-aee6-2c289b22c3ed" (UID: "9b66db9a-4de7-4eac-aee6-2c289b22c3ed"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:17:16.073135 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.073078    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-ca-bundle" pod "9b66db9a-4de7-4eac-aee6-2c289b22c3ed" (UID: "9b66db9a-4de7-4eac-aee6-2c289b22c3ed"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:17:16.074619 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.074594    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-storage-gcs-key" pod "9b66db9a-4de7-4eac-aee6-2c289b22c3ed" (UID: "9b66db9a-4de7-4eac-aee6-2c289b22c3ed"). InnerVolumeSpecName "storage-gcs-key". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:17:16.075004 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.074982    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-kube-api-access-xfqn4" pod "9b66db9a-4de7-4eac-aee6-2c289b22c3ed" (UID: "9b66db9a-4de7-4eac-aee6-2c289b22c3ed"). InnerVolumeSpecName "kube-api-access-xfqn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:17:16.075103 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.074985    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-querier-mtls" pod "9b66db9a-4de7-4eac-aee6-2c289b22c3ed" (UID: "9b66db9a-4de7-4eac-aee6-2c289b22c3ed"). InnerVolumeSpecName "tempo-data-science-tempostack-querier-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:17:16.174106 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.174044    4849 reconciler_common.go:299] "Volume detached for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-storage-gcs-key\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:16.174106 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.174070    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:16.174106 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.174085    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfqn4\" (UniqueName: \"kubernetes.io/projected/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-kube-api-access-xfqn4\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:16.174106 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.174098    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-querier-mtls\" (UniqueName: \"kubernetes.io/secret/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-data-science-tempostack-querier-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:16.174292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.174111    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/9b66db9a-4de7-4eac-aee6-2c289b22c3ed-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:16.735008 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.734976    4849 generic.go:356] "Generic (PLEG): container finished" podID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerID="254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933" exitCode=137
Oct 30 11:17:16.735452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.735041    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"
Oct 30 11:17:16.735452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.735038    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" event={"ID":"9b66db9a-4de7-4eac-aee6-2c289b22c3ed","Type":"ContainerDied","Data":"254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933"}
Oct 30 11:17:16.735452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.735209    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm" event={"ID":"9b66db9a-4de7-4eac-aee6-2c289b22c3ed","Type":"ContainerDied","Data":"41161bb280ec8b303227fb873b4b3301d40e3dbdaa20f4b75ef1fba07ad6b35a"}
Oct 30 11:17:16.735452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.735245    4849 scope.go:122] "RemoveContainer" containerID="254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933"
Oct 30 11:17:16.744660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.744643    4849 scope.go:122] "RemoveContainer" containerID="97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913"
Oct 30 11:17:16.753758 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.753744    4849 scope.go:122] "RemoveContainer" containerID="254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933"
Oct 30 11:17:16.754001 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:17:16.753985    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933\": container with ID starting with 254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933 not found: ID does not exist" containerID="254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933"
Oct 30 11:17:16.754076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.754008    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933"} err="failed to get container status \"254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933\": rpc error: code = NotFound desc = could not find container \"254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933\": container with ID starting with 254106e78d987939f0a507e0d50d3f1e35b80c0d950f093008b3251c63b60933 not found: ID does not exist"
Oct 30 11:17:16.754076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.754044    4849 scope.go:122] "RemoveContainer" containerID="97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913"
Oct 30 11:17:16.754241 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:17:16.754226    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913\": container with ID starting with 97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913 not found: ID does not exist" containerID="97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913"
Oct 30 11:17:16.754282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.754245    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913"} err="failed to get container status \"97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913\": rpc error: code = NotFound desc = could not find container \"97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913\": container with ID starting with 97a8b987ec2d79655f6fa9cc17b7541b01fae2ae7087263e377449675cfbb913 not found: ID does not exist"
Oct 30 11:17:16.791816 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.791797    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"]
Oct 30 11:17:16.804887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:16.804865    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-querier-5d44b8f5dc-nw8jm"]
Oct 30 11:17:18.753656 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:18.753620    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" path="/var/lib/kubelet/pods/9b66db9a-4de7-4eac-aee6-2c289b22c3ed/volumes"
Oct 30 11:17:25.459891 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:25.459846    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:17:29.752335 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:29.752300    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:17:35.136841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.136797    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:17:35.137309 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.137117    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="ec0ee2d4-0884-4dd2-b72a-897494c1b12a" containerName="otc-container" containerID="cri-o://8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11" gracePeriod=30
Oct 30 11:17:35.383735 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.383711    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:35.510184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.510097    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-monitoring-tls-certs\") pod \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") "
Oct 30 11:17:35.510184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.510184    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-kube-api-access-hmlkz\" (UniqueName: \"kubernetes.io/projected/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-kube-api-access-hmlkz\") pod \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") "
Oct 30 11:17:35.510402 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.510221    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-otc-internal\" (UniqueName: \"kubernetes.io/configmap/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-otc-internal\") pod \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\" (UID: \"ec0ee2d4-0884-4dd2-b72a-897494c1b12a\") "
Oct 30 11:17:35.510640 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.510610    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-otc-internal" pod "ec0ee2d4-0884-4dd2-b72a-897494c1b12a" (UID: "ec0ee2d4-0884-4dd2-b72a-897494c1b12a"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:17:35.512477 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.512445    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-monitoring-tls-certs" pod "ec0ee2d4-0884-4dd2-b72a-897494c1b12a" (UID: "ec0ee2d4-0884-4dd2-b72a-897494c1b12a"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:17:35.512632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.512600    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-kube-api-access-hmlkz" pod "ec0ee2d4-0884-4dd2-b72a-897494c1b12a" (UID: "ec0ee2d4-0884-4dd2-b72a-897494c1b12a"). InnerVolumeSpecName "kube-api-access-hmlkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:17:35.610973 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.610945    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmlkz\" (UniqueName: \"kubernetes.io/projected/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-kube-api-access-hmlkz\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:35.610973 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.610969    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:35.610973 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.610978    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec0ee2d4-0884-4dd2-b72a-897494c1b12a-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:17:35.839782 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.839751    4849 generic.go:356] "Generic (PLEG): container finished" podID="ec0ee2d4-0884-4dd2-b72a-897494c1b12a" containerID="8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11" exitCode=0
Oct 30 11:17:35.839942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.839822    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:35.839942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.839830    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ec0ee2d4-0884-4dd2-b72a-897494c1b12a","Type":"ContainerDied","Data":"8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11"}
Oct 30 11:17:35.839942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.839882    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ec0ee2d4-0884-4dd2-b72a-897494c1b12a","Type":"ContainerDied","Data":"5c271cb1f2f88905925fbfc30672fa35b30178ec49f004e6a03907bd3f726be4"}
Oct 30 11:17:35.839942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.839904    4849 scope.go:122] "RemoveContainer" containerID="8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11"
Oct 30 11:17:35.848903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.848887    4849 scope.go:122] "RemoveContainer" containerID="8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11"
Oct 30 11:17:35.849188 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:17:35.849168    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11\": container with ID starting with 8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11 not found: ID does not exist" containerID="8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11"
Oct 30 11:17:35.849253 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.849201    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11"} err="failed to get container status \"8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11\": rpc error: code = NotFound desc = could not find container \"8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11\": container with ID starting with 8c3a60af46350cef3c5b897d9c3e802af0743cd9dc350f14f2b5b0409f060e11 not found: ID does not exist"
Oct 30 11:17:35.894948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.894922    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:17:35.908123 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.908105    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:17:35.937225 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937205    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:17:35.937523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937511    4849 cpu_manager.go:414] "containerMap: removing container" podUID="ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" containerName="tempo"
Oct 30 11:17:35.937568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937524    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" containerName="tempo"
Oct 30 11:17:35.937568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937538    4849 cpu_manager.go:414] "containerMap: removing container" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo"
Oct 30 11:17:35.937568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937543    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo"
Oct 30 11:17:35.937568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937553    4849 cpu_manager.go:414] "containerMap: removing container" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo"
Oct 30 11:17:35.937568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937557    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo"
Oct 30 11:17:35.937568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937568    4849 cpu_manager.go:414] "containerMap: removing container" podUID="ec0ee2d4-0884-4dd2-b72a-897494c1b12a" containerName="otc-container"
Oct 30 11:17:35.937741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937573    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec0ee2d4-0884-4dd2-b72a-897494c1b12a" containerName="otc-container"
Oct 30 11:17:35.937741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937631    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef28dccc-28f1-4ab7-96ac-b7cf0ca4f902" containerName="tempo"
Oct 30 11:17:35.937741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937639    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo"
Oct 30 11:17:35.937741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937644    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b66db9a-4de7-4eac-aee6-2c289b22c3ed" containerName="tempo"
Oct 30 11:17:35.937741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.937651    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec0ee2d4-0884-4dd2-b72a-897494c1b12a" containerName="otc-container"
Oct 30 11:17:35.941899 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.941882    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:35.959294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.959274    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-0ed87044\""
Oct 30 11:17:35.959740 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.959709    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:17:35.959740 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.959738    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-789dw\""
Oct 30 11:17:35.962096 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:35.962077    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:17:36.015040 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.015009    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d580058f-f598-443c-a611-e7853cedf82d-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.015138 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.015064    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d580058f-f598-443c-a611-e7853cedf82d-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.015138 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.015081    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qqb8\" (UniqueName: \"kubernetes.io/projected/d580058f-f598-443c-a611-e7853cedf82d-kube-api-access-9qqb8\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.115965 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.115914    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d580058f-f598-443c-a611-e7853cedf82d-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.115965 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.115940    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qqb8\" (UniqueName: \"kubernetes.io/projected/d580058f-f598-443c-a611-e7853cedf82d-kube-api-access-9qqb8\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.116158 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.115987    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d580058f-f598-443c-a611-e7853cedf82d-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.116538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.116521    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d580058f-f598-443c-a611-e7853cedf82d-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.118401 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.118379    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d580058f-f598-443c-a611-e7853cedf82d-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.140719 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.140692    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qqb8\" (UniqueName: \"kubernetes.io/projected/d580058f-f598-443c-a611-e7853cedf82d-kube-api-access-9qqb8\") pod \"data-science-collector-collector-0\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.254056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.254036    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:17:36.553843 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.553819    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:17:36.555678 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:17:36.555652    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd580058f_f598_443c_a611_e7853cedf82d.slice/crio-a49ce1391ac5c5353f1cc8bf04ea5108ea95ec59792bcc52886439b177006e94 WatchSource:0}: Error finding container a49ce1391ac5c5353f1cc8bf04ea5108ea95ec59792bcc52886439b177006e94: Status 404 returned error can't find the container with id a49ce1391ac5c5353f1cc8bf04ea5108ea95ec59792bcc52886439b177006e94
Oct 30 11:17:36.557848 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.557819    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:17:36.557914 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.557874    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:17:36.753701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.753670    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec0ee2d4-0884-4dd2-b72a-897494c1b12a" path="/var/lib/kubelet/pods/ec0ee2d4-0884-4dd2-b72a-897494c1b12a/volumes"
Oct 30 11:17:36.846923 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.846892    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d580058f-f598-443c-a611-e7853cedf82d","Type":"ContainerStarted","Data":"01243786a8aadfd8708aa5b715057092844b903f60c6ff50504d356000a215d9"}
Oct 30 11:17:36.847100 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.846944    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d580058f-f598-443c-a611-e7853cedf82d","Type":"ContainerStarted","Data":"a49ce1391ac5c5353f1cc8bf04ea5108ea95ec59792bcc52886439b177006e94"}
Oct 30 11:17:36.892621 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:17:36.892580    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.89256813 podStartE2EDuration="1.89256813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:17:36.890671624 +0000 UTC m=+1954.271121062" watchObservedRunningTime="2026-10-30 11:17:36.89256813 +0000 UTC m=+1954.273017568"
Oct 30 11:17:39.430693 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:17:39.430656    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:18:06.779420 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:06.779384    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:18:06.779895 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:06.779588    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="d580058f-f598-443c-a611-e7853cedf82d" containerName="otc-container" containerID="cri-o://01243786a8aadfd8708aa5b715057092844b903f60c6ff50504d356000a215d9" gracePeriod=30
Oct 30 11:18:07.017266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.017239    4849 generic.go:356] "Generic (PLEG): container finished" podID="d580058f-f598-443c-a611-e7853cedf82d" containerID="01243786a8aadfd8708aa5b715057092844b903f60c6ff50504d356000a215d9" exitCode=0
Oct 30 11:18:07.017401 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.017288    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d580058f-f598-443c-a611-e7853cedf82d","Type":"ContainerDied","Data":"01243786a8aadfd8708aa5b715057092844b903f60c6ff50504d356000a215d9"}
Oct 30 11:18:07.035486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.035435    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:07.153747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.153720    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d580058f-f598-443c-a611-e7853cedf82d-otc-internal\" (UniqueName: \"kubernetes.io/configmap/d580058f-f598-443c-a611-e7853cedf82d-otc-internal\") pod \"d580058f-f598-443c-a611-e7853cedf82d\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") "
Oct 30 11:18:07.153878 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.153782    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d580058f-f598-443c-a611-e7853cedf82d-kube-api-access-9qqb8\" (UniqueName: \"kubernetes.io/projected/d580058f-f598-443c-a611-e7853cedf82d-kube-api-access-9qqb8\") pod \"d580058f-f598-443c-a611-e7853cedf82d\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") "
Oct 30 11:18:07.153919 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.153868    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d580058f-f598-443c-a611-e7853cedf82d-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d580058f-f598-443c-a611-e7853cedf82d-monitoring-tls-certs\") pod \"d580058f-f598-443c-a611-e7853cedf82d\" (UID: \"d580058f-f598-443c-a611-e7853cedf82d\") "
Oct 30 11:18:07.154142 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.154117    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d580058f-f598-443c-a611-e7853cedf82d-otc-internal" pod "d580058f-f598-443c-a611-e7853cedf82d" (UID: "d580058f-f598-443c-a611-e7853cedf82d"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:18:07.154243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.154175    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d580058f-f598-443c-a611-e7853cedf82d-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:07.155719 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.155698    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d580058f-f598-443c-a611-e7853cedf82d-monitoring-tls-certs" pod "d580058f-f598-443c-a611-e7853cedf82d" (UID: "d580058f-f598-443c-a611-e7853cedf82d"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:18:07.155719 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.155709    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d580058f-f598-443c-a611-e7853cedf82d-kube-api-access-9qqb8" pod "d580058f-f598-443c-a611-e7853cedf82d" (UID: "d580058f-f598-443c-a611-e7853cedf82d"). InnerVolumeSpecName "kube-api-access-9qqb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:18:07.255192 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.255167    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qqb8\" (UniqueName: \"kubernetes.io/projected/d580058f-f598-443c-a611-e7853cedf82d-kube-api-access-9qqb8\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:07.255192 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.255190    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d580058f-f598-443c-a611-e7853cedf82d-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:07.445149 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.445118    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"]
Oct 30 11:18:07.445339 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.445317    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo" containerID="cri-o://8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2" gracePeriod=30
Oct 30 11:18:07.457751 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.457720    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"]
Oct 30 11:18:07.457971 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.457946    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" containerID="cri-o://82ee2fc7f4dbf575f9447694daa939fd02b2742c977081be315182b2b367fa08" gracePeriod=30
Oct 30 11:18:07.707228 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:07.707138    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:18:08.023755 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:08.023677    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"d580058f-f598-443c-a611-e7853cedf82d","Type":"ContainerDied","Data":"a49ce1391ac5c5353f1cc8bf04ea5108ea95ec59792bcc52886439b177006e94"}
Oct 30 11:18:08.023755 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:08.023698    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:08.023755 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:08.023734    4849 scope.go:122] "RemoveContainer" containerID="01243786a8aadfd8708aa5b715057092844b903f60c6ff50504d356000a215d9"
Oct 30 11:18:08.081349 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:08.081314    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:18:08.093173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:08.093139    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:18:08.755033 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:08.754972    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d580058f-f598-443c-a611-e7853cedf82d" path="/var/lib/kubelet/pods/d580058f-f598-443c-a611-e7853cedf82d/volumes"
Oct 30 11:18:09.432370 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:18:09.432330    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:18:09.752296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:09.752223    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:18:17.471150 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:17.471102    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:18:19.752388 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:19.752336    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:18:27.568420 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:27.568369    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:18:27.568981 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:27.568513    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:18:28.130680 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.130647    4849 generic.go:356] "Generic (PLEG): container finished" podID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerID="82ee2fc7f4dbf575f9447694daa939fd02b2742c977081be315182b2b367fa08" exitCode=0
Oct 30 11:18:28.130843 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.130724    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" event={"ID":"aecc096f-f6a7-4105-9345-c2c6c3c5fce1","Type":"ContainerDied","Data":"82ee2fc7f4dbf575f9447694daa939fd02b2742c977081be315182b2b367fa08"}
Oct 30 11:18:28.230343 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.230291    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:18:28.308598 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.308572    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-tmp-storage\") pod \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") "
Oct 30 11:18:28.308727 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.308626    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-compactor-mtls\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-compactor-mtls\") pod \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") "
Oct 30 11:18:28.308727 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.308652    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-kube-api-access-wkmvc\" (UniqueName: \"kubernetes.io/projected/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-kube-api-access-wkmvc\") pod \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") "
Oct 30 11:18:28.308727 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.308683    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-conf\") pod \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") "
Oct 30 11:18:28.308910 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.308751    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-storage-gcs-key\") pod \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") "
Oct 30 11:18:28.308910 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.308780    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-ca-bundle\") pod \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\" (UID: \"aecc096f-f6a7-4105-9345-c2c6c3c5fce1\") "
Oct 30 11:18:28.309016 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.308953    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-tmp-storage" pod "aecc096f-f6a7-4105-9345-c2c6c3c5fce1" (UID: "aecc096f-f6a7-4105-9345-c2c6c3c5fce1"). InnerVolumeSpecName "tempo-tmp-storage". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:18:28.309141 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.309117    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-tmp-storage\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:28.309273 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.309251    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-conf" pod "aecc096f-f6a7-4105-9345-c2c6c3c5fce1" (UID: "aecc096f-f6a7-4105-9345-c2c6c3c5fce1"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:18:28.309345 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.309262    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-ca-bundle" pod "aecc096f-f6a7-4105-9345-c2c6c3c5fce1" (UID: "aecc096f-f6a7-4105-9345-c2c6c3c5fce1"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:18:28.310805 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.310783    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-compactor-mtls" pod "aecc096f-f6a7-4105-9345-c2c6c3c5fce1" (UID: "aecc096f-f6a7-4105-9345-c2c6c3c5fce1"). InnerVolumeSpecName "tempo-data-science-tempostack-compactor-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:18:28.310885 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.310787    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-kube-api-access-wkmvc" pod "aecc096f-f6a7-4105-9345-c2c6c3c5fce1" (UID: "aecc096f-f6a7-4105-9345-c2c6c3c5fce1"). InnerVolumeSpecName "kube-api-access-wkmvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:18:28.310885 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.310868    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-storage-gcs-key" pod "aecc096f-f6a7-4105-9345-c2c6c3c5fce1" (UID: "aecc096f-f6a7-4105-9345-c2c6c3c5fce1"). InnerVolumeSpecName "storage-gcs-key". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:18:28.410367 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.410345    4849 reconciler_common.go:299] "Volume detached for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-storage-gcs-key\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:28.410456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.410368    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:28.410456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.410379    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-compactor-mtls\" (UniqueName: \"kubernetes.io/secret/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-data-science-tempostack-compactor-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:28.410456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.410389    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkmvc\" (UniqueName: \"kubernetes.io/projected/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-kube-api-access-wkmvc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:28.410456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:28.410398    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/aecc096f-f6a7-4105-9345-c2c6c3c5fce1-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:29.137230 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:29.137201    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"
Oct 30 11:18:29.137642 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:29.137239    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd" event={"ID":"aecc096f-f6a7-4105-9345-c2c6c3c5fce1","Type":"ContainerDied","Data":"84d2ce0a58fcaf2a090f44b812aacbe9de28a75344d2d37fa03a1dcee3ea5c7a"}
Oct 30 11:18:29.137642 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:29.137297    4849 scope.go:122] "RemoveContainer" containerID="82ee2fc7f4dbf575f9447694daa939fd02b2742c977081be315182b2b367fa08"
Oct 30 11:18:29.192320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:29.192293    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"]
Oct 30 11:18:29.205714 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:29.205687    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-compactor-69bcb95f96-6brjd"]
Oct 30 11:18:30.752982 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:30.752932    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" path="/var/lib/kubelet/pods/aecc096f-f6a7-4105-9345-c2c6c3c5fce1/volumes"
Oct 30 11:18:35.140889 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.140839    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:18:35.141528 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.141452    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d580058f-f598-443c-a611-e7853cedf82d" containerName="otc-container"
Oct 30 11:18:35.141528 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.141472    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d580058f-f598-443c-a611-e7853cedf82d" containerName="otc-container"
Oct 30 11:18:35.141528 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.141501    4849 cpu_manager.go:414] "containerMap: removing container" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo"
Oct 30 11:18:35.141528 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.141509    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo"
Oct 30 11:18:35.141787 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.141612    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d580058f-f598-443c-a611-e7853cedf82d" containerName="otc-container"
Oct 30 11:18:35.141787 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.141627    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="aecc096f-f6a7-4105-9345-c2c6c3c5fce1" containerName="tempo"
Oct 30 11:18:35.146533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.146510    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.159718 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.159694    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.159830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.159728    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cxtz\" (UniqueName: \"kubernetes.io/projected/fbd0c8de-8f93-4672-915d-73cecca46151-kube-api-access-4cxtz\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.159830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.159749    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.159830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.159769    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbd0c8de-8f93-4672-915d-73cecca46151-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.159830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.159787    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.178331 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.178308    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-xn2fk\""
Oct 30 11:18:35.178471 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.178341    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:18:35.178471 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.178368    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:18:35.178557 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.178482    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:18:35.178664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.178650    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:18:35.180398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.180374    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:18:35.260846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.260817    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.261048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.260904    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbd0c8de-8f93-4672-915d-73cecca46151-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.261048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.260950    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.261183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.261052    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.261183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.261083    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cxtz\" (UniqueName: \"kubernetes.io/projected/fbd0c8de-8f93-4672-915d-73cecca46151-kube-api-access-4cxtz\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.261831 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.261802    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbd0c8de-8f93-4672-915d-73cecca46151-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.263370 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.263344    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.263486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.263375    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.263486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.263436    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.285951 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.285922    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cxtz\" (UniqueName: \"kubernetes.io/projected/fbd0c8de-8f93-4672-915d-73cecca46151-kube-api-access-4cxtz\") pod \"data-science-collector-collector-0\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.460230 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.460156    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:18:35.776872 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.776794    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:18:35.779942 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:18:35.779909    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd0c8de_8f93_4672_915d_73cecca46151.slice/crio-1df834b30b0516205155ebb7d44cf887e68edb4e82d906d4c2b623ec0dac395f WatchSource:0}: Error finding container 1df834b30b0516205155ebb7d44cf887e68edb4e82d906d4c2b623ec0dac395f: Status 404 returned error can't find the container with id 1df834b30b0516205155ebb7d44cf887e68edb4e82d906d4c2b623ec0dac395f
Oct 30 11:18:35.782146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.782113    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:18:35.782238 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:35.782172    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:18:36.176495 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:36.176456    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"fbd0c8de-8f93-4672-915d-73cecca46151","Type":"ContainerStarted","Data":"3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb"}
Oct 30 11:18:36.176895 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:36.176513    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"fbd0c8de-8f93-4672-915d-73cecca46151","Type":"ContainerStarted","Data":"1df834b30b0516205155ebb7d44cf887e68edb4e82d906d4c2b623ec0dac395f"}
Oct 30 11:18:36.223379 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:36.223322    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.223305306 podStartE2EDuration="1.223305306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:18:36.223127348 +0000 UTC m=+2013.603576784" watchObservedRunningTime="2026-10-30 11:18:36.223305306 +0000 UTC m=+2013.603754746"
Oct 30 11:18:37.466701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.466663    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo" probeResult="failure" output="Get \"https://10.132.0.82:3101/ready\": dial tcp 10.132.0.82:3101: connect: connection refused"
Oct 30 11:18:37.598018 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.597995    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:18:37.665821 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.665789    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"]
Oct 30 11:18:37.666194 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.666179    4849 cpu_manager.go:414] "containerMap: removing container" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo"
Oct 30 11:18:37.666244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.666195    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo"
Oct 30 11:18:37.666280 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.666265    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="381e80c9-06cb-4461-b710-e068648ba579" containerName="tempo"
Oct 30 11:18:37.669296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.669279    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"
Oct 30 11:18:37.679715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.679697    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-query-frontend-mtls\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-query-frontend-mtls\") pod \"381e80c9-06cb-4461-b710-e068648ba579\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") "
Oct 30 11:18:37.679813 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.679724    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-conf\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-conf\") pod \"381e80c9-06cb-4461-b710-e068648ba579\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") "
Oct 30 11:18:37.679813 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.679748    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-ca-bundle\") pod \"381e80c9-06cb-4461-b710-e068648ba579\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") "
Oct 30 11:18:37.679813 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.679793    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-storage-gcs-key\") pod \"381e80c9-06cb-4461-b710-e068648ba579\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") "
Oct 30 11:18:37.679985 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.679828    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/381e80c9-06cb-4461-b710-e068648ba579-kube-api-access-n96m7\" (UniqueName: \"kubernetes.io/projected/381e80c9-06cb-4461-b710-e068648ba579-kube-api-access-n96m7\") pod \"381e80c9-06cb-4461-b710-e068648ba579\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") "
Oct 30 11:18:37.679985 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.679902    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/381e80c9-06cb-4461-b710-e068648ba579-tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/381e80c9-06cb-4461-b710-e068648ba579-tempo-tmp-storage\") pod \"381e80c9-06cb-4461-b710-e068648ba579\" (UID: \"381e80c9-06cb-4461-b710-e068648ba579\") "
Oct 30 11:18:37.680223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.680198    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-ca-bundle" pod "381e80c9-06cb-4461-b710-e068648ba579" (UID: "381e80c9-06cb-4461-b710-e068648ba579"). InnerVolumeSpecName "tempo-data-science-tempostack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:18:37.680457 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.680412    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/381e80c9-06cb-4461-b710-e068648ba579-tempo-tmp-storage" pod "381e80c9-06cb-4461-b710-e068648ba579" (UID: "381e80c9-06cb-4461-b710-e068648ba579"). InnerVolumeSpecName "tempo-tmp-storage". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:18:37.680555 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.680450    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-conf" pod "381e80c9-06cb-4461-b710-e068648ba579" (UID: "381e80c9-06cb-4461-b710-e068648ba579"). InnerVolumeSpecName "tempo-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:18:37.682216 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.682185    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381e80c9-06cb-4461-b710-e068648ba579-kube-api-access-n96m7" pod "381e80c9-06cb-4461-b710-e068648ba579" (UID: "381e80c9-06cb-4461-b710-e068648ba579"). InnerVolumeSpecName "kube-api-access-n96m7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:18:37.682337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.682313    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-query-frontend-mtls" pod "381e80c9-06cb-4461-b710-e068648ba579" (UID: "381e80c9-06cb-4461-b710-e068648ba579"). InnerVolumeSpecName "tempo-data-science-tempostack-query-frontend-mtls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:18:37.682422 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.682408    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-storage-gcs-key" pod "381e80c9-06cb-4461-b710-e068648ba579" (UID: "381e80c9-06cb-4461-b710-e068648ba579"). InnerVolumeSpecName "storage-gcs-key". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:18:37.686799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.686704    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-kh8sh\""
Oct 30 11:18:37.689803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.689782    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"]
Oct 30 11:18:37.780739 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.780670    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxwn5\" (UniqueName: \"kubernetes.io/projected/3e34af22-5522-43d8-b7cd-5f2d4067d51d-kube-api-access-wxwn5\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch\" (UID: \"3e34af22-5522-43d8-b7cd-5f2d4067d51d\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"
Oct 30 11:18:37.780849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.780755    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-query-frontend-mtls\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-query-frontend-mtls\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:37.780849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.780804    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-conf\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-conf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:37.780849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.780830    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-data-science-tempostack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381e80c9-06cb-4461-b710-e068648ba579-tempo-data-science-tempostack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:37.780849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.780844    4849 reconciler_common.go:299] "Volume detached for volume \"storage-gcs-key\" (UniqueName: \"kubernetes.io/secret/381e80c9-06cb-4461-b710-e068648ba579-storage-gcs-key\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:37.781000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.780858    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n96m7\" (UniqueName: \"kubernetes.io/projected/381e80c9-06cb-4461-b710-e068648ba579-kube-api-access-n96m7\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:37.781000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.780871    4849 reconciler_common.go:299] "Volume detached for volume \"tempo-tmp-storage\" (UniqueName: \"kubernetes.io/empty-dir/381e80c9-06cb-4461-b710-e068648ba579-tempo-tmp-storage\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:18:37.881519 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.881493    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxwn5\" (UniqueName: \"kubernetes.io/projected/3e34af22-5522-43d8-b7cd-5f2d4067d51d-kube-api-access-wxwn5\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch\" (UID: \"3e34af22-5522-43d8-b7cd-5f2d4067d51d\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"
Oct 30 11:18:37.906062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.906038    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxwn5\" (UniqueName: \"kubernetes.io/projected/3e34af22-5522-43d8-b7cd-5f2d4067d51d-kube-api-access-wxwn5\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch\" (UID: \"3e34af22-5522-43d8-b7cd-5f2d4067d51d\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"
Oct 30 11:18:37.981204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:37.981186    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"
Oct 30 11:18:38.187368 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.187335    4849 generic.go:356] "Generic (PLEG): container finished" podID="381e80c9-06cb-4461-b710-e068648ba579" containerID="8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2" exitCode=137
Oct 30 11:18:38.187523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.187381    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"
Oct 30 11:18:38.187523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.187394    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" event={"ID":"381e80c9-06cb-4461-b710-e068648ba579","Type":"ContainerDied","Data":"8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2"}
Oct 30 11:18:38.187523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.187434    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l" event={"ID":"381e80c9-06cb-4461-b710-e068648ba579","Type":"ContainerDied","Data":"4699e79d8daf634472f482ebec6a24fe0d443690c15ae1132a744d165d8ffe38"}
Oct 30 11:18:38.187523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.187455    4849 scope.go:122] "RemoveContainer" containerID="8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2"
Oct 30 11:18:38.196066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.196016    4849 scope.go:122] "RemoveContainer" containerID="8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2"
Oct 30 11:18:38.196347 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:18:38.196329    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2\": container with ID starting with 8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2 not found: ID does not exist" containerID="8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2"
Oct 30 11:18:38.196406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.196356    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2"} err="failed to get container status \"8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2\": rpc error: code = NotFound desc = could not find container \"8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2\": container with ID starting with 8e5b934cd80d1af1207cbbf0c147b092c55a5249bc6d5f237e6e1c50db3015f2 not found: ID does not exist"
Oct 30 11:18:38.243000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.242977    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"]
Oct 30 11:18:38.246829 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.246810    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/tempo-data-science-tempostack-query-frontend-8447b5d879-tmh6l"]
Oct 30 11:18:38.281061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.281043    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"]
Oct 30 11:18:38.282889 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:18:38.282844    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e34af22_5522_43d8_b7cd_5f2d4067d51d.slice/crio-259f086391457d4c98795d505486250da41ba92be18f243903ea95c56018a22f WatchSource:0}: Error finding container 259f086391457d4c98795d505486250da41ba92be18f243903ea95c56018a22f: Status 404 returned error can't find the container with id 259f086391457d4c98795d505486250da41ba92be18f243903ea95c56018a22f
Oct 30 11:18:38.551760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.551664    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:18:38.556767 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.556746    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.557798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.557769    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:18:38.562243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.562221    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.577461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.577442    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 11:18:38.577789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.577775    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 11:18:38.594124 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.594097    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:18:38.607852 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.607823    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:18:38.642799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.642780    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 11:18:38.642799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.642787    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 11:18:38.642983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.642844    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:18:38.642983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.642890    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 11:18:38.642983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.642931    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 11:18:38.642983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.642965    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 11:18:38.643211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.642994    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 11:18:38.643211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.643105    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 11:18:38.643211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.643143    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:18:38.643211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.643157    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 11:18:38.643211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.643199    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-jrn7f\""
Oct 30 11:18:38.643370 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.643217    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-k7dqn\""
Oct 30 11:18:38.677715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.677695    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 11:18:38.687215 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687193    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687307 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687227    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687307 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687251    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-tls-assets\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.687421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687403    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687451    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687474    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687491    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzzqt\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-kube-api-access-mzzqt\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.687567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687547    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687600 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687568    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.687644 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687629    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687678 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687663    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687718 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687698    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-config-volume\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.687759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687723    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.687759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687749    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-config-out\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.687834 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687786    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-web-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.687887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687833    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687857    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687916    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.687977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.687953    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5vt4\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-kube-api-access-b5vt4\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.753428 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.753389    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="381e80c9-06cb-4461-b710-e068648ba579" path="/var/lib/kubelet/pods/381e80c9-06cb-4461-b710-e068648ba579/volumes"
Oct 30 11:18:38.788704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788680    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-config-out\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.788796 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788708    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-web-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.788796 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788743    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.788884 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788858    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.788923 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788894    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.788977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788924    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5vt4\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-kube-api-access-b5vt4\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.788977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788971    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.788997    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789043    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-tls-assets\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.789155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789097    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789129    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789154    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789178    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzzqt\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-kube-api-access-mzzqt\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.789414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789217    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789246    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.789414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789281    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789327    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.789414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789354    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-config-volume\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.789414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789378    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.789784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.789757    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.790317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.790285    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.790416 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.790341    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.790871 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.790830    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.792157 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.792102    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-web-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.792157 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.792102    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.792836 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.792808    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-config-out\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.794066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.793815    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.794066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.793869    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.794066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.793957    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.794066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.793987    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.794350 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.794288    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-config-volume\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.794350 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.794329    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.795139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.795120    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-tls-assets\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.795320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.795301    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.795320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.795310    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.795591 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.795569    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.814114 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.814097    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzzqt\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-kube-api-access-mzzqt\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:38.815496 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.815478    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5vt4\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-kube-api-access-b5vt4\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:38.887198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:38.887168    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:39.196939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:39.196864    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch" event={"ID":"3e34af22-5522-43d8-b7cd-5f2d4067d51d","Type":"ContainerStarted","Data":"ee030ff55238d024fd04ce422b8fa6803ee7a06fc0e13c85dea92453115428b9"}
Oct 30 11:18:39.196939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:39.196909    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch" event={"ID":"3e34af22-5522-43d8-b7cd-5f2d4067d51d","Type":"ContainerStarted","Data":"259f086391457d4c98795d505486250da41ba92be18f243903ea95c56018a22f"}
Oct 30 11:18:39.222550 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:39.222478    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:18:39.224751 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:18:39.224723    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c756993_cd82_48eb_837e_219a7e989822.slice/crio-e59f59bcec26d2872944aeb4ba5c6a4a1ee53691c6211f781cf17e1f1b5e928e WatchSource:0}: Error finding container e59f59bcec26d2872944aeb4ba5c6a4a1ee53691c6211f781cf17e1f1b5e928e: Status 404 returned error can't find the container with id e59f59bcec26d2872944aeb4ba5c6a4a1ee53691c6211f781cf17e1f1b5e928e
Oct 30 11:18:39.243190 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:39.243137    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch" podStartSLOduration=2.243124431 podStartE2EDuration="2.243124431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:18:39.24133873 +0000 UTC m=+2016.621788170" watchObservedRunningTime="2026-10-30 11:18:39.243124431 +0000 UTC m=+2016.623573869"
Oct 30 11:18:39.429968 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:18:39.429926    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:18:40.204676 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:40.204642    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerStarted","Data":"b3ed1ade347c013c8679f498947038e03ba63150e60f617b76ac2966742a6212"}
Oct 30 11:18:40.205063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:40.204712    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerStarted","Data":"e59f59bcec26d2872944aeb4ba5c6a4a1ee53691c6211f781cf17e1f1b5e928e"}
Oct 30 11:18:43.221776 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:43.221703    4849 generic.go:356] "Generic (PLEG): container finished" podID="8c756993-cd82-48eb-837e-219a7e989822" containerID="b3ed1ade347c013c8679f498947038e03ba63150e60f617b76ac2966742a6212" exitCode=0
Oct 30 11:18:43.222105 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:43.221786    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerDied","Data":"b3ed1ade347c013c8679f498947038e03ba63150e60f617b76ac2966742a6212"}
Oct 30 11:18:44.229442 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.229410    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerStarted","Data":"775899c8e815a61269f2b82ffd07afda3467467754d7ee0fadbd5b415c9a9fd1"}
Oct 30 11:18:44.229792 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.229474    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerStarted","Data":"d373a858bd79d5e8e8114b8adb6a2160b1a10cec5467874df2b5af4ba3d93d7c"}
Oct 30 11:18:44.229792 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.229651    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:44.309811 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.309769    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podStartSLOduration=3.124316597 podStartE2EDuration="6.309757587s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.18544099s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:18:44.308333894 +0000 UTC m=+2021.688783456" watchObservedRunningTime="2026-10-30 11:18:44.309757587 +0000 UTC m=+2021.690207063"
Oct 30 11:18:44.834348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.834321    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:44.867054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.866710    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:44.935453 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.935428    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:44.935589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.935570    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:44.951109 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:44.951085    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:45.054308 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:45.054285    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5f4e377c3e259c071aab429e31ca61846b879cc3476e2f02b70e888ccdcdb590/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:45.081920 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:45.081898    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:45.179800 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:45.179742    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:18:45.519007 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:45.518978    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:18:45.520517 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:18:45.520490    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c7935a_f2e7_4546_9f2b_62c4ee5b63c1.slice/crio-ef83ef05e9dac4d6c92134e6624a0925b5065278854537d66d369387ceff6a24 WatchSource:0}: Error finding container ef83ef05e9dac4d6c92134e6624a0925b5065278854537d66d369387ceff6a24: Status 404 returned error can't find the container with id ef83ef05e9dac4d6c92134e6624a0925b5065278854537d66d369387ceff6a24
Oct 30 11:18:46.242438 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:46.242370    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerStarted","Data":"4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98"}
Oct 30 11:18:46.242438 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:46.242427    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerStarted","Data":"ef83ef05e9dac4d6c92134e6624a0925b5065278854537d66d369387ceff6a24"}
Oct 30 11:18:50.238130 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:50.238099    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:18:50.265692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:50.265660    4849 generic.go:356] "Generic (PLEG): container finished" podID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerID="4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98" exitCode=0
Oct 30 11:18:50.265846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:50.265742    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerDied","Data":"4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98"}
Oct 30 11:18:51.273493 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:51.273453    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerStarted","Data":"5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66"}
Oct 30 11:18:51.273921 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:51.273544    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerStarted","Data":"b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95"}
Oct 30 11:18:51.273921 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:51.273586    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerStarted","Data":"5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0"}
Oct 30 11:18:51.330124 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:51.330082    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podStartSLOduration=9.944265985 podStartE2EDuration="13.330068893s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.385802908s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:18:51.328743647 +0000 UTC m=+2028.709193086" watchObservedRunningTime="2026-10-30 11:18:51.330068893 +0000 UTC m=+2028.710518331"
Oct 30 11:18:55.180395 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:18:55.180355    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:19:00.180131 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:00.180090    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:19:00.185922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:00.185892    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:19:00.329778 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:00.329753    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:19:09.429585 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:19:09.429550    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:19:11.606129 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:11.606097    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:11.606621 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:11.606372    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="fbd0c8de-8f93-4672-915d-73cecca46151" containerName="otc-container" containerID="cri-o://3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb" gracePeriod=30
Oct 30 11:19:13.968889 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:13.968864    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.068137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.068110    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbd0c8de-8f93-4672-915d-73cecca46151-kube-api-access-4cxtz\" (UniqueName: \"kubernetes.io/projected/fbd0c8de-8f93-4672-915d-73cecca46151-kube-api-access-4cxtz\") pod \"fbd0c8de-8f93-4672-915d-73cecca46151\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") "
Oct 30 11:19:14.068295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.068150    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-tls-certs\") pod \"fbd0c8de-8f93-4672-915d-73cecca46151\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") "
Oct 30 11:19:14.068295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.068196    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-monitoring-tls-certs\") pod \"fbd0c8de-8f93-4672-915d-73cecca46151\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") "
Oct 30 11:19:14.068295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.068227    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbd0c8de-8f93-4672-915d-73cecca46151-otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbd0c8de-8f93-4672-915d-73cecca46151-otc-internal\") pod \"fbd0c8de-8f93-4672-915d-73cecca46151\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") "
Oct 30 11:19:14.068295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.068253    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-data-science-collector-ta-client-cert\") pod \"fbd0c8de-8f93-4672-915d-73cecca46151\" (UID: \"fbd0c8de-8f93-4672-915d-73cecca46151\") "
Oct 30 11:19:14.068689 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.068652    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbd0c8de-8f93-4672-915d-73cecca46151-otc-internal" pod "fbd0c8de-8f93-4672-915d-73cecca46151" (UID: "fbd0c8de-8f93-4672-915d-73cecca46151"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:19:14.070404 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.070376    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-tls-certs" pod "fbd0c8de-8f93-4672-915d-73cecca46151" (UID: "fbd0c8de-8f93-4672-915d-73cecca46151"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:14.070502 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.070420    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd0c8de-8f93-4672-915d-73cecca46151-kube-api-access-4cxtz" pod "fbd0c8de-8f93-4672-915d-73cecca46151" (UID: "fbd0c8de-8f93-4672-915d-73cecca46151"). InnerVolumeSpecName "kube-api-access-4cxtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:19:14.070502 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.070430    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-data-science-collector-ta-client-cert" pod "fbd0c8de-8f93-4672-915d-73cecca46151" (UID: "fbd0c8de-8f93-4672-915d-73cecca46151"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:14.070502 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.070460    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-monitoring-tls-certs" pod "fbd0c8de-8f93-4672-915d-73cecca46151" (UID: "fbd0c8de-8f93-4672-915d-73cecca46151"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:14.169457 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.169434    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cxtz\" (UniqueName: \"kubernetes.io/projected/fbd0c8de-8f93-4672-915d-73cecca46151-kube-api-access-4cxtz\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:14.169457 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.169457    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:14.169587 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.169466    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:14.169587 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.169475    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbd0c8de-8f93-4672-915d-73cecca46151-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:14.169587 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.169485    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/fbd0c8de-8f93-4672-915d-73cecca46151-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:14.395627 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.395568    4849 generic.go:356] "Generic (PLEG): container finished" podID="fbd0c8de-8f93-4672-915d-73cecca46151" containerID="3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb" exitCode=0
Oct 30 11:19:14.395627 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.395616    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.395772 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.395654    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"fbd0c8de-8f93-4672-915d-73cecca46151","Type":"ContainerDied","Data":"3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb"}
Oct 30 11:19:14.395772 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.395729    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"fbd0c8de-8f93-4672-915d-73cecca46151","Type":"ContainerDied","Data":"1df834b30b0516205155ebb7d44cf887e68edb4e82d906d4c2b623ec0dac395f"}
Oct 30 11:19:14.395772 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.395752    4849 scope.go:122] "RemoveContainer" containerID="3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb"
Oct 30 11:19:14.404701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.404687    4849 scope.go:122] "RemoveContainer" containerID="3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb"
Oct 30 11:19:14.404954 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:19:14.404937    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb\": container with ID starting with 3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb not found: ID does not exist" containerID="3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb"
Oct 30 11:19:14.405005 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.404961    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb"} err="failed to get container status \"3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb\": rpc error: code = NotFound desc = could not find container \"3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb\": container with ID starting with 3af84dcc858fcd57a8e90b488e077f0350f57dfaebb5288e6a28df714cbe5afb not found: ID does not exist"
Oct 30 11:19:14.451180 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.451157    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:14.464654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.464633    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:14.493012 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.492990    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:14.493387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.493374    4849 cpu_manager.go:414] "containerMap: removing container" podUID="fbd0c8de-8f93-4672-915d-73cecca46151" containerName="otc-container"
Oct 30 11:19:14.493439 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.493388    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbd0c8de-8f93-4672-915d-73cecca46151" containerName="otc-container"
Oct 30 11:19:14.493487 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.493477    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbd0c8de-8f93-4672-915d-73cecca46151" containerName="otc-container"
Oct 30 11:19:14.497888 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.497865    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.514987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.514894    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-xn2fk\""
Oct 30 11:19:14.514987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.514952    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:19:14.514987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.514950    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:19:14.515215 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.515097    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:19:14.515215 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.515156    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:19:14.516650 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.516626    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:14.572782 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.572757    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.572873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.572795    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzb7q\" (UniqueName: \"kubernetes.io/projected/4afc0480-2782-4fc1-86fa-cd7ddadc607f-kube-api-access-jzb7q\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.572873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.572828    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.572949 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.572925    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.572986 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.572973    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4afc0480-2782-4fc1-86fa-cd7ddadc607f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.674303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.674244    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.674408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.674310    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.674408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.674365    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4afc0480-2782-4fc1-86fa-cd7ddadc607f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.674408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.674400    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.674581 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.674437    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzb7q\" (UniqueName: \"kubernetes.io/projected/4afc0480-2782-4fc1-86fa-cd7ddadc607f-kube-api-access-jzb7q\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.675191 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.675165    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4afc0480-2782-4fc1-86fa-cd7ddadc607f-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.676815 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.676794    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.677282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.677262    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.677376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.677298    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.709633 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.709611    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzb7q\" (UniqueName: \"kubernetes.io/projected/4afc0480-2782-4fc1-86fa-cd7ddadc607f-kube-api-access-jzb7q\") pod \"data-science-collector-collector-0\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:14.753624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.753601    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbd0c8de-8f93-4672-915d-73cecca46151" path="/var/lib/kubelet/pods/fbd0c8de-8f93-4672-915d-73cecca46151/volumes"
Oct 30 11:19:14.809951 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:14.809930    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:15.111287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:15.111265    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:15.113576 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:19:15.113549    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4afc0480_2782_4fc1_86fa_cd7ddadc607f.slice/crio-34e8c05f52b36ef1bb219c896e59c4e553a0948b4ea4649196a10727b8746b4d WatchSource:0}: Error finding container 34e8c05f52b36ef1bb219c896e59c4e553a0948b4ea4649196a10727b8746b4d: Status 404 returned error can't find the container with id 34e8c05f52b36ef1bb219c896e59c4e553a0948b4ea4649196a10727b8746b4d
Oct 30 11:19:15.115708 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:15.115671    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:19:15.115785 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:15.115735    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:19:15.403063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:15.402955    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4afc0480-2782-4fc1-86fa-cd7ddadc607f","Type":"ContainerStarted","Data":"ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560"}
Oct 30 11:19:15.403063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:15.403012    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4afc0480-2782-4fc1-86fa-cd7ddadc607f","Type":"ContainerStarted","Data":"34e8c05f52b36ef1bb219c896e59c4e553a0948b4ea4649196a10727b8746b4d"}
Oct 30 11:19:15.449282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:15.449235    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.449221664 podStartE2EDuration="1.449221664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:19:15.447892463 +0000 UTC m=+2052.828341895" watchObservedRunningTime="2026-10-30 11:19:15.449221664 +0000 UTC m=+2052.829671301"
Oct 30 11:19:39.429651 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:19:39.429613    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:19:44.756517 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:44.756488    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:44.757043 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:44.756664    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="4afc0480-2782-4fc1-86fa-cd7ddadc607f" containerName="otc-container" containerID="cri-o://ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560" gracePeriod=30
Oct 30 11:19:47.104745 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.104721    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.117261 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.117229    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4afc0480-2782-4fc1-86fa-cd7ddadc607f-kube-api-access-jzb7q\" (UniqueName: \"kubernetes.io/projected/4afc0480-2782-4fc1-86fa-cd7ddadc607f-kube-api-access-jzb7q\") pod \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") "
Oct 30 11:19:47.117365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.117310    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-monitoring-tls-certs\") pod \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") "
Oct 30 11:19:47.117365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.117359    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-tls-certs\") pod \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") "
Oct 30 11:19:47.117452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.117419    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-data-science-collector-ta-client-cert\") pod \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") "
Oct 30 11:19:47.117497 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.117458    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4afc0480-2782-4fc1-86fa-cd7ddadc607f-otc-internal\" (UniqueName: \"kubernetes.io/configmap/4afc0480-2782-4fc1-86fa-cd7ddadc607f-otc-internal\") pod \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\" (UID: \"4afc0480-2782-4fc1-86fa-cd7ddadc607f\") "
Oct 30 11:19:47.117927 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.117897    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4afc0480-2782-4fc1-86fa-cd7ddadc607f-otc-internal" pod "4afc0480-2782-4fc1-86fa-cd7ddadc607f" (UID: "4afc0480-2782-4fc1-86fa-cd7ddadc607f"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:19:47.119315 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.119291    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afc0480-2782-4fc1-86fa-cd7ddadc607f-kube-api-access-jzb7q" pod "4afc0480-2782-4fc1-86fa-cd7ddadc607f" (UID: "4afc0480-2782-4fc1-86fa-cd7ddadc607f"). InnerVolumeSpecName "kube-api-access-jzb7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:19:47.119432 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.119411    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-monitoring-tls-certs" pod "4afc0480-2782-4fc1-86fa-cd7ddadc607f" (UID: "4afc0480-2782-4fc1-86fa-cd7ddadc607f"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:47.119470 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.119426    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-tls-certs" pod "4afc0480-2782-4fc1-86fa-cd7ddadc607f" (UID: "4afc0480-2782-4fc1-86fa-cd7ddadc607f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:47.119507 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.119496    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-data-science-collector-ta-client-cert" pod "4afc0480-2782-4fc1-86fa-cd7ddadc607f" (UID: "4afc0480-2782-4fc1-86fa-cd7ddadc607f"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:47.218536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.218473    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:47.218536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.218500    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:47.218536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.218513    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/4afc0480-2782-4fc1-86fa-cd7ddadc607f-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:47.218536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.218522    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/4afc0480-2782-4fc1-86fa-cd7ddadc607f-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:47.218536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.218531    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzb7q\" (UniqueName: \"kubernetes.io/projected/4afc0480-2782-4fc1-86fa-cd7ddadc607f-kube-api-access-jzb7q\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:47.596185 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.596147    4849 generic.go:356] "Generic (PLEG): container finished" podID="4afc0480-2782-4fc1-86fa-cd7ddadc607f" containerID="ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560" exitCode=0
Oct 30 11:19:47.596307 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.596199    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.596307 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.596232    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4afc0480-2782-4fc1-86fa-cd7ddadc607f","Type":"ContainerDied","Data":"ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560"}
Oct 30 11:19:47.596307 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.596285    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"4afc0480-2782-4fc1-86fa-cd7ddadc607f","Type":"ContainerDied","Data":"34e8c05f52b36ef1bb219c896e59c4e553a0948b4ea4649196a10727b8746b4d"}
Oct 30 11:19:47.596307 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.596307    4849 scope.go:122] "RemoveContainer" containerID="ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560"
Oct 30 11:19:47.605944 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.605927    4849 scope.go:122] "RemoveContainer" containerID="ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560"
Oct 30 11:19:47.606244 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:19:47.606226    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560\": container with ID starting with ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560 not found: ID does not exist" containerID="ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560"
Oct 30 11:19:47.606299 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.606253    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560"} err="failed to get container status \"ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560\": rpc error: code = NotFound desc = could not find container \"ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560\": container with ID starting with ac0443a8a1a0ff3ddbdceede51892dcc85ecada6460b7eb0927c880805c47560 not found: ID does not exist"
Oct 30 11:19:47.652015 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.651992    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:47.665314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.665293    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:47.691699 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.691675    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:47.692270 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.692251    4849 cpu_manager.go:414] "containerMap: removing container" podUID="4afc0480-2782-4fc1-86fa-cd7ddadc607f" containerName="otc-container"
Oct 30 11:19:47.692328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.692274    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4afc0480-2782-4fc1-86fa-cd7ddadc607f" containerName="otc-container"
Oct 30 11:19:47.692376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.692364    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="4afc0480-2782-4fc1-86fa-cd7ddadc607f" containerName="otc-container"
Oct 30 11:19:47.700604 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.700586    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.715885 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.715859    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:47.717223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.717202    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:19:47.717223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.717216    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-7c92b5b1\""
Oct 30 11:19:47.717377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.717270    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-xn2fk\""
Oct 30 11:19:47.717377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.717285    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:19:47.717377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.717318    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:19:47.722646 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.722614    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.722776 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.722671    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.722846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.722797    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a499e819-49d2-4c5b-bf7e-271e164f08e4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.722846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.722820    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh77k\" (UniqueName: \"kubernetes.io/projected/a499e819-49d2-4c5b-bf7e-271e164f08e4-kube-api-access-vh77k\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.722972 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.722848    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.823416 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.823386    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.823535 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.823437    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a499e819-49d2-4c5b-bf7e-271e164f08e4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.823535 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.823452    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh77k\" (UniqueName: \"kubernetes.io/projected/a499e819-49d2-4c5b-bf7e-271e164f08e4-kube-api-access-vh77k\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.823664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.823528    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.823664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.823641    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.824209 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.824185    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a499e819-49d2-4c5b-bf7e-271e164f08e4-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.825967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.825940    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.825967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.825952    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.826120 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.825987    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:47.852539 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:47.850965    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh77k\" (UniqueName: \"kubernetes.io/projected/a499e819-49d2-4c5b-bf7e-271e164f08e4-kube-api-access-vh77k\") pod \"data-science-collector-collector-0\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:48.013469 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.013445    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:48.313455 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.313428    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:48.315406 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:19:48.315376    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda499e819_49d2_4c5b_bf7e_271e164f08e4.slice/crio-a3ca4c1e326053fbe17c3c436a79f919424e0d0bafb4ac463e737651c4b0ac2f WatchSource:0}: Error finding container a3ca4c1e326053fbe17c3c436a79f919424e0d0bafb4ac463e737651c4b0ac2f: Status 404 returned error can't find the container with id a3ca4c1e326053fbe17c3c436a79f919424e0d0bafb4ac463e737651c4b0ac2f
Oct 30 11:19:48.317625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.317588    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:19:48.317728 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.317660    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:19:48.604077 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.603980    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a499e819-49d2-4c5b-bf7e-271e164f08e4","Type":"ContainerStarted","Data":"b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1"}
Oct 30 11:19:48.604077 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.604064    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a499e819-49d2-4c5b-bf7e-271e164f08e4","Type":"ContainerStarted","Data":"a3ca4c1e326053fbe17c3c436a79f919424e0d0bafb4ac463e737651c4b0ac2f"}
Oct 30 11:19:48.649869 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.649824    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.649811468 podStartE2EDuration="1.649811468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:19:48.647712141 +0000 UTC m=+2086.028161581" watchObservedRunningTime="2026-10-30 11:19:48.649811468 +0000 UTC m=+2086.030260907"
Oct 30 11:19:48.754151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:48.754116    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4afc0480-2782-4fc1-86fa-cd7ddadc607f" path="/var/lib/kubelet/pods/4afc0480-2782-4fc1-86fa-cd7ddadc607f/volumes"
Oct 30 11:19:51.795595 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:51.795558    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:51.796081 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:51.795827    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="a499e819-49d2-4c5b-bf7e-271e164f08e4" containerName="otc-container" containerID="cri-o://b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1" gracePeriod=30
Oct 30 11:19:52.043009 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.042983    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.059613 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.059595    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a499e819-49d2-4c5b-bf7e-271e164f08e4-kube-api-access-vh77k\" (UniqueName: \"kubernetes.io/projected/a499e819-49d2-4c5b-bf7e-271e164f08e4-kube-api-access-vh77k\") pod \"a499e819-49d2-4c5b-bf7e-271e164f08e4\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") "
Oct 30 11:19:52.059720 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.059642    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-tls-certs\") pod \"a499e819-49d2-4c5b-bf7e-271e164f08e4\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") "
Oct 30 11:19:52.059720 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.059674    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a499e819-49d2-4c5b-bf7e-271e164f08e4-otc-internal\" (UniqueName: \"kubernetes.io/configmap/a499e819-49d2-4c5b-bf7e-271e164f08e4-otc-internal\") pod \"a499e819-49d2-4c5b-bf7e-271e164f08e4\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") "
Oct 30 11:19:52.059720 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.059708    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-monitoring-tls-certs\") pod \"a499e819-49d2-4c5b-bf7e-271e164f08e4\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") "
Oct 30 11:19:52.059888 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.059741    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-data-science-collector-ta-client-cert\") pod \"a499e819-49d2-4c5b-bf7e-271e164f08e4\" (UID: \"a499e819-49d2-4c5b-bf7e-271e164f08e4\") "
Oct 30 11:19:52.060098 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.060057    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a499e819-49d2-4c5b-bf7e-271e164f08e4-otc-internal" pod "a499e819-49d2-4c5b-bf7e-271e164f08e4" (UID: "a499e819-49d2-4c5b-bf7e-271e164f08e4"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:19:52.061666 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.061620    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a499e819-49d2-4c5b-bf7e-271e164f08e4-kube-api-access-vh77k" pod "a499e819-49d2-4c5b-bf7e-271e164f08e4" (UID: "a499e819-49d2-4c5b-bf7e-271e164f08e4"). InnerVolumeSpecName "kube-api-access-vh77k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:19:52.061947 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.061930    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-data-science-collector-ta-client-cert" pod "a499e819-49d2-4c5b-bf7e-271e164f08e4" (UID: "a499e819-49d2-4c5b-bf7e-271e164f08e4"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:52.062221 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.062202    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-tls-certs" pod "a499e819-49d2-4c5b-bf7e-271e164f08e4" (UID: "a499e819-49d2-4c5b-bf7e-271e164f08e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:52.062300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.062262    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-monitoring-tls-certs" pod "a499e819-49d2-4c5b-bf7e-271e164f08e4" (UID: "a499e819-49d2-4c5b-bf7e-271e164f08e4"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:19:52.160664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.160625    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh77k\" (UniqueName: \"kubernetes.io/projected/a499e819-49d2-4c5b-bf7e-271e164f08e4-kube-api-access-vh77k\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:52.160664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.160649    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:52.160664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.160661    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a499e819-49d2-4c5b-bf7e-271e164f08e4-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:52.160664 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.160669    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:52.161002 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.160678    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a499e819-49d2-4c5b-bf7e-271e164f08e4-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:19:52.633589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.633556    4849 generic.go:356] "Generic (PLEG): container finished" podID="a499e819-49d2-4c5b-bf7e-271e164f08e4" containerID="b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1" exitCode=0
Oct 30 11:19:52.633742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.633601    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.633742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.633639    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a499e819-49d2-4c5b-bf7e-271e164f08e4","Type":"ContainerDied","Data":"b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1"}
Oct 30 11:19:52.633742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.633693    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a499e819-49d2-4c5b-bf7e-271e164f08e4","Type":"ContainerDied","Data":"a3ca4c1e326053fbe17c3c436a79f919424e0d0bafb4ac463e737651c4b0ac2f"}
Oct 30 11:19:52.633742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.633717    4849 scope.go:122] "RemoveContainer" containerID="b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1"
Oct 30 11:19:52.642837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.642815    4849 scope.go:122] "RemoveContainer" containerID="b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1"
Oct 30 11:19:52.643115 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:19:52.643096    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1\": container with ID starting with b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1 not found: ID does not exist" containerID="b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1"
Oct 30 11:19:52.643199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.643127    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1"} err="failed to get container status \"b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1\": rpc error: code = NotFound desc = could not find container \"b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1\": container with ID starting with b1938b7035c7bcceba4646c39a58b0876555798761aa894af374d570c39f12f1 not found: ID does not exist"
Oct 30 11:19:52.688942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.688919    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:52.702198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.702175    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:52.727067 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.727049    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:52.727381 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.727370    4849 cpu_manager.go:414] "containerMap: removing container" podUID="a499e819-49d2-4c5b-bf7e-271e164f08e4" containerName="otc-container"
Oct 30 11:19:52.727423 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.727382    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a499e819-49d2-4c5b-bf7e-271e164f08e4" containerName="otc-container"
Oct 30 11:19:52.727480 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.727470    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="a499e819-49d2-4c5b-bf7e-271e164f08e4" containerName="otc-container"
Oct 30 11:19:52.734945 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.734932    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.754054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.754005    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:19:52.754187 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.754056    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-xn2fk\""
Oct 30 11:19:52.754187 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.754178    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:19:52.754335 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.754167    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a499e819-49d2-4c5b-bf7e-271e164f08e4" path="/var/lib/kubelet/pods/a499e819-49d2-4c5b-bf7e-271e164f08e4/volumes"
Oct 30 11:19:52.754335 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.754173    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:19:52.754335 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.754096    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:19:52.754591 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.754575    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:52.765240 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.765220    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.765355 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.765253    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.765355 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.765283    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.765499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.765376    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.765499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.765455    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhlps\" (UniqueName: \"kubernetes.io/projected/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-kube-api-access-hhlps\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.866071 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.866048    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.866071 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.866077    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.866519 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.866098    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhlps\" (UniqueName: \"kubernetes.io/projected/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-kube-api-access-hhlps\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.866519 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.866220    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.866519 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.866273    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.866850 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.866830    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.868574 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.868546    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.868841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.868818    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.868881 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.868849    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:52.890304 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:52.890259    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhlps\" (UniqueName: \"kubernetes.io/projected/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-kube-api-access-hhlps\") pod \"data-science-collector-collector-0\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:53.047365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:53.047340    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:19:53.351967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:53.351943    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:53.354160 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:19:53.354131    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode434dccb_2d8a_4866_a3a4_e4b3d8ab32d3.slice/crio-5146f46e5828a9a53052ad855a4c06569a873a403b98efe48c9c45541d8ca4e8 WatchSource:0}: Error finding container 5146f46e5828a9a53052ad855a4c06569a873a403b98efe48c9c45541d8ca4e8: Status 404 returned error can't find the container with id 5146f46e5828a9a53052ad855a4c06569a873a403b98efe48c9c45541d8ca4e8
Oct 30 11:19:53.356345 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:53.356312    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:19:53.356422 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:53.356372    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:19:53.640903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:53.640818    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3","Type":"ContainerStarted","Data":"124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3"}
Oct 30 11:19:53.640903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:53.640871    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3","Type":"ContainerStarted","Data":"5146f46e5828a9a53052ad855a4c06569a873a403b98efe48c9c45541d8ca4e8"}
Oct 30 11:19:53.685646 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:53.685603    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.685590728 podStartE2EDuration="1.685590728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:19:53.684811821 +0000 UTC m=+2091.065261254" watchObservedRunningTime="2026-10-30 11:19:53.685590728 +0000 UTC m=+2091.066040166"
Oct 30 11:19:59.840630 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:59.840596    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:19:59.841146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:19:59.840784    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" containerName="otc-container" containerID="cri-o://124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3" gracePeriod=30
Oct 30 11:20:00.086747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.086722    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.120969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.120915    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-monitoring-tls-certs\") pod \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") "
Oct 30 11:20:00.120969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.120964    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-tls-certs\") pod \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") "
Oct 30 11:20:00.121189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.120985    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-kube-api-access-hhlps\" (UniqueName: \"kubernetes.io/projected/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-kube-api-access-hhlps\") pod \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") "
Oct 30 11:20:00.121189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.121003    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-data-science-collector-ta-client-cert\") pod \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") "
Oct 30 11:20:00.121189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.121075    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-otc-internal\" (UniqueName: \"kubernetes.io/configmap/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-otc-internal\") pod \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\" (UID: \"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3\") "
Oct 30 11:20:00.121591 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.121556    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-otc-internal" pod "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" (UID: "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:20:00.123134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.123107    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-data-science-collector-ta-client-cert" pod "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" (UID: "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:00.123224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.123130    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-kube-api-access-hhlps" pod "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" (UID: "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3"). InnerVolumeSpecName "kube-api-access-hhlps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:20:00.123224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.123145    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-tls-certs" pod "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" (UID: "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:00.123224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.123130    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-monitoring-tls-certs" pod "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" (UID: "e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:00.222538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.222508    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhlps\" (UniqueName: \"kubernetes.io/projected/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-kube-api-access-hhlps\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:00.222538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.222533    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:00.222538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.222544    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:00.222739 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.222553    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:00.222739 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.222561    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:00.680842 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.680808    4849 generic.go:356] "Generic (PLEG): container finished" podID="e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" containerID="124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3" exitCode=0
Oct 30 11:20:00.680971 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.680858    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.680971 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.680882    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3","Type":"ContainerDied","Data":"124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3"}
Oct 30 11:20:00.680971 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.680929    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3","Type":"ContainerDied","Data":"5146f46e5828a9a53052ad855a4c06569a873a403b98efe48c9c45541d8ca4e8"}
Oct 30 11:20:00.680971 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.680954    4849 scope.go:122] "RemoveContainer" containerID="124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3"
Oct 30 11:20:00.691097 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.691079    4849 scope.go:122] "RemoveContainer" containerID="124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3"
Oct 30 11:20:00.694528 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:00.691700    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3\": container with ID starting with 124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3 not found: ID does not exist" containerID="124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3"
Oct 30 11:20:00.694528 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.691735    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3"} err="failed to get container status \"124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3\": rpc error: code = NotFound desc = could not find container \"124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3\": container with ID starting with 124c789784cae0ed79f75ac8f90e4b0e619091a618a2d8b71255b8f6189eeed3 not found: ID does not exist"
Oct 30 11:20:00.734821 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.734799    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:00.753561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.753538    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:00.778322 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.778298    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:00.778670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.778657    4849 cpu_manager.go:414] "containerMap: removing container" podUID="e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" containerName="otc-container"
Oct 30 11:20:00.778715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.778672    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" containerName="otc-container"
Oct 30 11:20:00.778763 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.778753    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" containerName="otc-container"
Oct 30 11:20:00.783099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.783085    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.801695 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.801677    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:00.802716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.802699    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:20:00.802845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.802717    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-xn2fk\""
Oct 30 11:20:00.802845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.802728    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:20:00.802845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.802758    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:20:00.827484 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.827459    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj5k2\" (UniqueName: \"kubernetes.io/projected/5b19f28c-8222-4eec-9909-6ee7c7a28d88-kube-api-access-qj5k2\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.827673 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.827512    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b19f28c-8222-4eec-9909-6ee7c7a28d88-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.827673 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.827630    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.827763 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.827745    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.827796 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.827780    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.892188 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.892168    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-7c92b5b1\""
Oct 30 11:20:00.928845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.928825    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj5k2\" (UniqueName: \"kubernetes.io/projected/5b19f28c-8222-4eec-9909-6ee7c7a28d88-kube-api-access-qj5k2\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.928917 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.928854    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b19f28c-8222-4eec-9909-6ee7c7a28d88-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.928917 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.928874    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.929000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.928926    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.929000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.928949    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.929570 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.929548    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b19f28c-8222-4eec-9909-6ee7c7a28d88-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.931466 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.931421    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.931637 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.931617    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.931680 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.931623    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:00.952928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:00.952906    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj5k2\" (UniqueName: \"kubernetes.io/projected/5b19f28c-8222-4eec-9909-6ee7c7a28d88-kube-api-access-qj5k2\") pod \"data-science-collector-collector-0\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:01.095183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:01.095159    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:01.394346 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:01.394320    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:01.396078 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:20:01.396050    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b19f28c_8222_4eec_9909_6ee7c7a28d88.slice/crio-eb34a37fa705430dd7bb3c5c3a0681d7744b8b90ecd7f31b9e9359072339e161 WatchSource:0}: Error finding container eb34a37fa705430dd7bb3c5c3a0681d7744b8b90ecd7f31b9e9359072339e161: Status 404 returned error can't find the container with id eb34a37fa705430dd7bb3c5c3a0681d7744b8b90ecd7f31b9e9359072339e161
Oct 30 11:20:01.398204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:01.398173    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:20:01.398295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:01.398247    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:20:01.695076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:01.694977    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b19f28c-8222-4eec-9909-6ee7c7a28d88","Type":"ContainerStarted","Data":"1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8"}
Oct 30 11:20:01.695076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:01.695066    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b19f28c-8222-4eec-9909-6ee7c7a28d88","Type":"ContainerStarted","Data":"eb34a37fa705430dd7bb3c5c3a0681d7744b8b90ecd7f31b9e9359072339e161"}
Oct 30 11:20:01.740928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:01.740871    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.740854562 podStartE2EDuration="1.740854562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:20:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:20:01.73887355 +0000 UTC m=+2099.119322989" watchObservedRunningTime="2026-10-30 11:20:01.740854562 +0000 UTC m=+2099.121304002"
Oct 30 11:20:02.753611 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:02.753566    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3" path="/var/lib/kubelet/pods/e434dccb-2d8a-4866-a3a4-e4b3d8ab32d3/volumes"
Oct 30 11:20:02.983912 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:02.983816    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:20:03.016538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:02.992198    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:20:09.431842 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:09.431802    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:20:09.899054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:09.898998    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:09.899313 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:09.899273    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="5b19f28c-8222-4eec-9909-6ee7c7a28d88" containerName="otc-container" containerID="cri-o://1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8" gracePeriod=30
Oct 30 11:20:10.161917 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.161864    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:10.200834 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.200805    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-data-science-collector-ta-client-cert\") pod \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") "
Oct 30 11:20:10.200955 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.200844    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b19f28c-8222-4eec-9909-6ee7c7a28d88-otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b19f28c-8222-4eec-9909-6ee7c7a28d88-otc-internal\") pod \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") "
Oct 30 11:20:10.200955 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.200908    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-monitoring-tls-certs\") pod \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") "
Oct 30 11:20:10.200955 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.200949    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-tls-certs\") pod \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") "
Oct 30 11:20:10.201111 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.200978    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b19f28c-8222-4eec-9909-6ee7c7a28d88-kube-api-access-qj5k2\" (UniqueName: \"kubernetes.io/projected/5b19f28c-8222-4eec-9909-6ee7c7a28d88-kube-api-access-qj5k2\") pod \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\" (UID: \"5b19f28c-8222-4eec-9909-6ee7c7a28d88\") "
Oct 30 11:20:10.201280 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.201249    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b19f28c-8222-4eec-9909-6ee7c7a28d88-otc-internal" pod "5b19f28c-8222-4eec-9909-6ee7c7a28d88" (UID: "5b19f28c-8222-4eec-9909-6ee7c7a28d88"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:20:10.203136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.203100    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-tls-certs" pod "5b19f28c-8222-4eec-9909-6ee7c7a28d88" (UID: "5b19f28c-8222-4eec-9909-6ee7c7a28d88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:10.203379 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.203352    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-monitoring-tls-certs" pod "5b19f28c-8222-4eec-9909-6ee7c7a28d88" (UID: "5b19f28c-8222-4eec-9909-6ee7c7a28d88"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:10.203450 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.203405    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b19f28c-8222-4eec-9909-6ee7c7a28d88-kube-api-access-qj5k2" pod "5b19f28c-8222-4eec-9909-6ee7c7a28d88" (UID: "5b19f28c-8222-4eec-9909-6ee7c7a28d88"). InnerVolumeSpecName "kube-api-access-qj5k2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:20:10.203525 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.203446    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-data-science-collector-ta-client-cert" pod "5b19f28c-8222-4eec-9909-6ee7c7a28d88" (UID: "5b19f28c-8222-4eec-9909-6ee7c7a28d88"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:10.302226 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.302191    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:10.302226 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.302220    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:10.302412 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.302233    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj5k2\" (UniqueName: \"kubernetes.io/projected/5b19f28c-8222-4eec-9909-6ee7c7a28d88-kube-api-access-qj5k2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:10.302412 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.302248    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5b19f28c-8222-4eec-9909-6ee7c7a28d88-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:10.302412 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.302261    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5b19f28c-8222-4eec-9909-6ee7c7a28d88-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:10.740830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.740797    4849 generic.go:356] "Generic (PLEG): container finished" podID="5b19f28c-8222-4eec-9909-6ee7c7a28d88" containerID="1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8" exitCode=0
Oct 30 11:20:10.741262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.740848    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:10.741262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.740854    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b19f28c-8222-4eec-9909-6ee7c7a28d88","Type":"ContainerDied","Data":"1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8"}
Oct 30 11:20:10.741262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.740900    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5b19f28c-8222-4eec-9909-6ee7c7a28d88","Type":"ContainerDied","Data":"eb34a37fa705430dd7bb3c5c3a0681d7744b8b90ecd7f31b9e9359072339e161"}
Oct 30 11:20:10.741262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.740929    4849 scope.go:122] "RemoveContainer" containerID="1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8"
Oct 30 11:20:10.751823 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.751806    4849 scope.go:122] "RemoveContainer" containerID="1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8"
Oct 30 11:20:10.752128 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:10.752106    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8\": container with ID starting with 1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8 not found: ID does not exist" containerID="1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8"
Oct 30 11:20:10.752189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.752138    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8"} err="failed to get container status \"1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8\": rpc error: code = NotFound desc = could not find container \"1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8\": container with ID starting with 1ba9c49a7f2f21566a56311d9f75048bd60e5e636cc4c1ae35d009941ddcc0a8 not found: ID does not exist"
Oct 30 11:20:10.796787 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.796762    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:10.810381 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:10.810354    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:11.404813 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.404784    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"]
Oct 30 11:20:11.404991 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.404970    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch" podUID="3e34af22-5522-43d8-b7cd-5f2d4067d51d" containerName="thanos-querier" containerID="cri-o://ee030ff55238d024fd04ce422b8fa6803ee7a06fc0e13c85dea92453115428b9" gracePeriod=30
Oct 30 11:20:11.604584 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.604553    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:20:11.605138 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.604925    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="prometheus" containerID="cri-o://5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0" gracePeriod=600
Oct 30 11:20:11.605138 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.605120    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="config-reloader" containerID="cri-o://b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95" gracePeriod=600
Oct 30 11:20:11.606464 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.606418    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="thanos-sidecar" containerID="cri-o://5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66" gracePeriod=600
Oct 30 11:20:11.628627 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.628603    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:20:11.628875 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.628853    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="alertmanager" containerID="cri-o://d373a858bd79d5e8e8114b8adb6a2160b1a10cec5467874df2b5af4ba3d93d7c" gracePeriod=120
Oct 30 11:20:11.629069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.629016    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="config-reloader" containerID="cri-o://775899c8e815a61269f2b82ffd07afda3467467754d7ee0fadbd5b415c9a9fd1" gracePeriod=120
Oct 30 11:20:11.749303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.749279    4849 generic.go:356] "Generic (PLEG): container finished" podID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerID="5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66" exitCode=0
Oct 30 11:20:11.749303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.749301    4849 generic.go:356] "Generic (PLEG): container finished" podID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerID="5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0" exitCode=0
Oct 30 11:20:11.749704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.749347    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerDied","Data":"5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66"}
Oct 30 11:20:11.749704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.749418    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerDied","Data":"5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0"}
Oct 30 11:20:11.750674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.750656    4849 generic.go:356] "Generic (PLEG): container finished" podID="3e34af22-5522-43d8-b7cd-5f2d4067d51d" containerID="ee030ff55238d024fd04ce422b8fa6803ee7a06fc0e13c85dea92453115428b9" exitCode=0
Oct 30 11:20:11.750745 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.750698    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch" event={"ID":"3e34af22-5522-43d8-b7cd-5f2d4067d51d","Type":"ContainerDied","Data":"ee030ff55238d024fd04ce422b8fa6803ee7a06fc0e13c85dea92453115428b9"}
Oct 30 11:20:11.835583 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.835556    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"
Oct 30 11:20:11.905043 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.905007    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:20:11.914929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.914907    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e34af22-5522-43d8-b7cd-5f2d4067d51d-kube-api-access-wxwn5\" (UniqueName: \"kubernetes.io/projected/3e34af22-5522-43d8-b7cd-5f2d4067d51d-kube-api-access-wxwn5\") pod \"3e34af22-5522-43d8-b7cd-5f2d4067d51d\" (UID: \"3e34af22-5522-43d8-b7cd-5f2d4067d51d\") "
Oct 30 11:20:11.916804 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:11.916779    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e34af22-5522-43d8-b7cd-5f2d4067d51d-kube-api-access-wxwn5" pod "3e34af22-5522-43d8-b7cd-5f2d4067d51d" (UID: "3e34af22-5522-43d8-b7cd-5f2d4067d51d"). InnerVolumeSpecName "kube-api-access-wxwn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:20:12.015803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.015731    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.015956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.015852    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.015956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.015900    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016105 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.015980    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016105 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016049    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016105 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016083    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config-out\" (UniqueName: \"kubernetes.io/empty-dir/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config-out\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016125    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-tls-assets\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-tls-assets\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016180    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-kube-api-access-b5vt4\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-kube-api-access-b5vt4\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016238    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-thanos-prometheus-http-client-file\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016305    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-secret-prometheus-web-tls-ca\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016336    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016337    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-1" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:20:12.016436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016352    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-0" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:20:12.016436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016367    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016417    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config\") pod \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\" (UID: \"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1\") "
Oct 30 11:20:12.016747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016708    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.016747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016730    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.016871 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.016747    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxwn5\" (UniqueName: \"kubernetes.io/projected/3e34af22-5522-43d8-b7cd-5f2d4067d51d-kube-api-access-wxwn5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.018499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.018469    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:12.019288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.018664    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:12.019288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.018799    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-2" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:20:12.019801 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.019776    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-secret-prometheus-web-tls-ca" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:12.020184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.020161    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:12.020326 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.020311    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-thanos-prometheus-http-client-file" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:12.020936 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.020906    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-kube-api-access-b5vt4" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "kube-api-access-b5vt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:20:12.021067 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.020966    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-tls-assets" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:20:12.021183 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.021166    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config-out" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:20:12.022240 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.022221    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:20:12.032919 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.032892    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config" pod "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" (UID: "85c7935a-f2e7-4546-9f2b-62c4ee5b63c1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:12.117870 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117844    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.117870 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117867    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5vt4\" (UniqueName: \"kubernetes.io/projected/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-kube-api-access-b5vt4\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117883    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117896    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117910    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117924    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117938    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117951    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.117986    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.118002    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.118061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.118017    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:12.129327 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.129311    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:20:12.218895 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.218870    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\""
Oct 30 11:20:12.760605 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.760517    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b19f28c-8222-4eec-9909-6ee7c7a28d88" path="/var/lib/kubelet/pods/5b19f28c-8222-4eec-9909-6ee7c7a28d88/volumes"
Oct 30 11:20:12.766079 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.766051    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"
Oct 30 11:20:12.766211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.766069    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch" event={"ID":"3e34af22-5522-43d8-b7cd-5f2d4067d51d","Type":"ContainerDied","Data":"259f086391457d4c98795d505486250da41ba92be18f243903ea95c56018a22f"}
Oct 30 11:20:12.766211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.766125    4849 scope.go:122] "RemoveContainer" containerID="ee030ff55238d024fd04ce422b8fa6803ee7a06fc0e13c85dea92453115428b9"
Oct 30 11:20:12.769354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.769330    4849 generic.go:356] "Generic (PLEG): container finished" podID="8c756993-cd82-48eb-837e-219a7e989822" containerID="775899c8e815a61269f2b82ffd07afda3467467754d7ee0fadbd5b415c9a9fd1" exitCode=0
Oct 30 11:20:12.769472 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.769455    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerDied","Data":"775899c8e815a61269f2b82ffd07afda3467467754d7ee0fadbd5b415c9a9fd1"}
Oct 30 11:20:12.772277 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.772255    4849 generic.go:356] "Generic (PLEG): container finished" podID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerID="b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95" exitCode=0
Oct 30 11:20:12.772391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.772299    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerDied","Data":"b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95"}
Oct 30 11:20:12.772391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.772353    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"85c7935a-f2e7-4546-9f2b-62c4ee5b63c1","Type":"ContainerDied","Data":"ef83ef05e9dac4d6c92134e6624a0925b5065278854537d66d369387ceff6a24"}
Oct 30 11:20:12.772391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.772362    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:20:12.778647 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.778631    4849 scope.go:122] "RemoveContainer" containerID="5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66"
Oct 30 11:20:12.786642 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.786617    4849 scope.go:122] "RemoveContainer" containerID="b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95"
Oct 30 11:20:12.796726 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.796711    4849 scope.go:122] "RemoveContainer" containerID="5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0"
Oct 30 11:20:12.804901 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.804880    4849 scope.go:122] "RemoveContainer" containerID="4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98"
Oct 30 11:20:12.812855 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.812836    4849 scope.go:122] "RemoveContainer" containerID="5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66"
Oct 30 11:20:12.813150 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:12.813130    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66\": container with ID starting with 5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66 not found: ID does not exist" containerID="5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66"
Oct 30 11:20:12.813203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.813158    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66"} err="failed to get container status \"5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66\": rpc error: code = NotFound desc = could not find container \"5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66\": container with ID starting with 5a1c04d4e3729ef8b9eccbbc79238f72a2e10dc1a22ebcf2d57333c85aac7d66 not found: ID does not exist"
Oct 30 11:20:12.813203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.813175    4849 scope.go:122] "RemoveContainer" containerID="b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95"
Oct 30 11:20:12.813447 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:12.813424    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95\": container with ID starting with b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95 not found: ID does not exist" containerID="b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95"
Oct 30 11:20:12.813561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.813454    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95"} err="failed to get container status \"b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95\": rpc error: code = NotFound desc = could not find container \"b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95\": container with ID starting with b5427e1f966730d88220bf11b582d667973bc30d3cdc29bc2eec074f2c332b95 not found: ID does not exist"
Oct 30 11:20:12.813561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.813469    4849 scope.go:122] "RemoveContainer" containerID="5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0"
Oct 30 11:20:12.813698 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:12.813684    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0\": container with ID starting with 5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0 not found: ID does not exist" containerID="5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0"
Oct 30 11:20:12.813734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.813700    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0"} err="failed to get container status \"5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0\": rpc error: code = NotFound desc = could not find container \"5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0\": container with ID starting with 5290b6a1e31506f11dd0cdbda6f2166c6a36fe96f7aba3346a4def79306305d0 not found: ID does not exist"
Oct 30 11:20:12.813734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.813712    4849 scope.go:122] "RemoveContainer" containerID="4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98"
Oct 30 11:20:12.813977 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:12.813944    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98\": container with ID starting with 4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98 not found: ID does not exist" containerID="4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98"
Oct 30 11:20:12.814100 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.813978    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98"} err="failed to get container status \"4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98\": rpc error: code = NotFound desc = could not find container \"4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98\": container with ID starting with 4b2404922f183ce34191c276fa996a39e0490caa8abe53e24f2127151bc12a98 not found: ID does not exist"
Oct 30 11:20:12.848575 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.848550    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:20:12.873204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.873169    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:20:12.922766 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.922739    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"]
Oct 30 11:20:12.935956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:12.935933    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-ttkch"]
Oct 30 11:20:14.754014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:14.753922    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e34af22-5522-43d8-b7cd-5f2d4067d51d" path="/var/lib/kubelet/pods/3e34af22-5522-43d8-b7cd-5f2d4067d51d/volumes"
Oct 30 11:20:14.754479 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:14.754435    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" path="/var/lib/kubelet/pods/85c7935a-f2e7-4546-9f2b-62c4ee5b63c1/volumes"
Oct 30 11:20:15.236960 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:15.236928    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.88:9093/-/ready\": dial tcp 10.132.0.88:9093: connect: connection refused"
Oct 30 11:20:19.814385 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814353    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814684    4849 cpu_manager.go:414] "containerMap: removing container" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="init-config-reloader"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814695    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="init-config-reloader"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814724    4849 cpu_manager.go:414] "containerMap: removing container" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="config-reloader"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814729    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="config-reloader"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814740    4849 cpu_manager.go:414] "containerMap: removing container" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="prometheus"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814745    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="prometheus"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814751    4849 cpu_manager.go:414] "containerMap: removing container" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="thanos-sidecar"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814755    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="thanos-sidecar"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814767    4849 cpu_manager.go:414] "containerMap: removing container" podUID="3e34af22-5522-43d8-b7cd-5f2d4067d51d" containerName="thanos-querier"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814771    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e34af22-5522-43d8-b7cd-5f2d4067d51d" containerName="thanos-querier"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814779    4849 cpu_manager.go:414] "containerMap: removing container" podUID="5b19f28c-8222-4eec-9909-6ee7c7a28d88" containerName="otc-container"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814783    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b19f28c-8222-4eec-9909-6ee7c7a28d88" containerName="otc-container"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814836    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="config-reloader"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814843    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="prometheus"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814850    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="85c7935a-f2e7-4546-9f2b-62c4ee5b63c1" containerName="thanos-sidecar"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814857    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e34af22-5522-43d8-b7cd-5f2d4067d51d" containerName="thanos-querier"
Oct 30 11:20:19.814976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.814863    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b19f28c-8222-4eec-9909-6ee7c7a28d88" containerName="otc-container"
Oct 30 11:20:19.819511 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.819489    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.838114 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.838091    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:19.849233 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.849207    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:20:19.852169 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.852147    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-rcknf\""
Oct 30 11:20:19.862762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.862736    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:20:19.862866 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.862808    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:20:19.862866 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.862827    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:20:19.875710 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.875455    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhzsz\" (UniqueName: \"kubernetes.io/projected/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-kube-api-access-rhzsz\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.875710 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.875503    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.875710 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.875547    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.875710 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.875579    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.875710 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.875602    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.976254 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.976215    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhzsz\" (UniqueName: \"kubernetes.io/projected/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-kube-api-access-rhzsz\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.976449 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.976269    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.976449 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.976333    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.976840 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.976770    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.976840 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.976818    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.977388 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.977366    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.979882 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.979792    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.980279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.980247    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:19.980625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:19.980604    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:20.000308 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.000285    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhzsz\" (UniqueName: \"kubernetes.io/projected/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-kube-api-access-rhzsz\") pod \"data-science-collector-collector-0\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:20.134314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.134254    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:20:20.237611 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.237572    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.88:9093/-/ready\": dial tcp 10.132.0.88:9093: connect: connection refused"
Oct 30 11:20:20.438820 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.438743    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:20:20.441699 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:20:20.441673    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ffb0d0_4afa_448e_9164_56a4d6ecefcf.slice/crio-c0de4a1b9b3b0e32258374277701b041581977994ab739f21348f4f06a364f62 WatchSource:0}: Error finding container c0de4a1b9b3b0e32258374277701b041581977994ab739f21348f4f06a364f62: Status 404 returned error can't find the container with id c0de4a1b9b3b0e32258374277701b041581977994ab739f21348f4f06a364f62
Oct 30 11:20:20.443749 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.443720    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:20:20.443810 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.443780    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:20:20.821872 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.821842    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"71ffb0d0-4afa-448e-9164-56a4d6ecefcf","Type":"ContainerStarted","Data":"95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe"}
Oct 30 11:20:20.822252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.821893    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"71ffb0d0-4afa-448e-9164-56a4d6ecefcf","Type":"ContainerStarted","Data":"c0de4a1b9b3b0e32258374277701b041581977994ab739f21348f4f06a364f62"}
Oct 30 11:20:20.867961 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:20.867911    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.867894777 podStartE2EDuration="1.867894777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:20:20.866559388 +0000 UTC m=+2118.247008828" watchObservedRunningTime="2026-10-30 11:20:20.867894777 +0000 UTC m=+2118.248344216"
Oct 30 11:20:21.836672 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.836637    4849 generic.go:356] "Generic (PLEG): container finished" podID="8c756993-cd82-48eb-837e-219a7e989822" containerID="d373a858bd79d5e8e8114b8adb6a2160b1a10cec5467874df2b5af4ba3d93d7c" exitCode=0
Oct 30 11:20:21.837113 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.836697    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerDied","Data":"d373a858bd79d5e8e8114b8adb6a2160b1a10cec5467874df2b5af4ba3d93d7c"}
Oct 30 11:20:21.883960 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.883941    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:20:21.994240 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994177    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-web-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-web-config\") pod \"8c756993-cd82-48eb-837e-219a7e989822\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") "
Oct 30 11:20:21.994240 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994211    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-alertmanager-data-science-monitoringstack-db\") pod \"8c756993-cd82-48eb-837e-219a7e989822\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") "
Oct 30 11:20:21.994408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994268    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-config-out\") pod \"8c756993-cd82-48eb-837e-219a7e989822\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") "
Oct 30 11:20:21.994408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994299    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-tls-assets\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-tls-assets\") pod \"8c756993-cd82-48eb-837e-219a7e989822\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") "
Oct 30 11:20:21.994408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994333    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-kube-api-access-mzzqt\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-kube-api-access-mzzqt\") pod \"8c756993-cd82-48eb-837e-219a7e989822\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") "
Oct 30 11:20:21.994408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994363    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-cluster-tls-config\") pod \"8c756993-cd82-48eb-837e-219a7e989822\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") "
Oct 30 11:20:21.994408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994392    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-config-volume\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-config-volume\") pod \"8c756993-cd82-48eb-837e-219a7e989822\" (UID: \"8c756993-cd82-48eb-837e-219a7e989822\") "
Oct 30 11:20:21.994674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.994632    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-alertmanager-data-science-monitoringstack-db" pod "8c756993-cd82-48eb-837e-219a7e989822" (UID: "8c756993-cd82-48eb-837e-219a7e989822"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:20:21.996718 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.996689    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-kube-api-access-mzzqt" pod "8c756993-cd82-48eb-837e-219a7e989822" (UID: "8c756993-cd82-48eb-837e-219a7e989822"). InnerVolumeSpecName "kube-api-access-mzzqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:20:21.996873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.996855    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-tls-assets" pod "8c756993-cd82-48eb-837e-219a7e989822" (UID: "8c756993-cd82-48eb-837e-219a7e989822"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:20:21.996930 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.996865    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-config-out" pod "8c756993-cd82-48eb-837e-219a7e989822" (UID: "8c756993-cd82-48eb-837e-219a7e989822"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:20:21.996998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:21.996974    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-config-volume" pod "8c756993-cd82-48eb-837e-219a7e989822" (UID: "8c756993-cd82-48eb-837e-219a7e989822"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:22.002149 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.002130    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-cluster-tls-config" pod "8c756993-cd82-48eb-837e-219a7e989822" (UID: "8c756993-cd82-48eb-837e-219a7e989822"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:22.008879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.008856    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-web-config" pod "8c756993-cd82-48eb-837e-219a7e989822" (UID: "8c756993-cd82-48eb-837e-219a7e989822"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:20:22.096033 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.095999    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:22.096119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.096044    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:22.096119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.096055    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzzqt\" (UniqueName: \"kubernetes.io/projected/8c756993-cd82-48eb-837e-219a7e989822-kube-api-access-mzzqt\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:22.096119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.096064    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:22.096119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.096072    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:22.096119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.096091    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c756993-cd82-48eb-837e-219a7e989822-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:22.096119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.096100    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/8c756993-cd82-48eb-837e-219a7e989822-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:20:22.843302 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.843272    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:20:22.843302 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.843281    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"8c756993-cd82-48eb-837e-219a7e989822","Type":"ContainerDied","Data":"e59f59bcec26d2872944aeb4ba5c6a4a1ee53691c6211f781cf17e1f1b5e928e"}
Oct 30 11:20:22.843715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.843344    4849 scope.go:122] "RemoveContainer" containerID="775899c8e815a61269f2b82ffd07afda3467467754d7ee0fadbd5b415c9a9fd1"
Oct 30 11:20:22.851722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.851705    4849 scope.go:122] "RemoveContainer" containerID="d373a858bd79d5e8e8114b8adb6a2160b1a10cec5467874df2b5af4ba3d93d7c"
Oct 30 11:20:22.859302 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.859285    4849 scope.go:122] "RemoveContainer" containerID="b3ed1ade347c013c8679f498947038e03ba63150e60f617b76ac2966742a6212"
Oct 30 11:20:22.912771 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.912752    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:20:22.926731 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:22.926701    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:20:23.112977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.112948    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:20:23.113288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113272    4849 cpu_manager.go:414] "containerMap: removing container" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="config-reloader"
Oct 30 11:20:23.113288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113287    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="config-reloader"
Oct 30 11:20:23.113454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113312    4849 cpu_manager.go:414] "containerMap: removing container" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="init-config-reloader"
Oct 30 11:20:23.113454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113317    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="init-config-reloader"
Oct 30 11:20:23.113454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113329    4849 cpu_manager.go:414] "containerMap: removing container" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="alertmanager"
Oct 30 11:20:23.113454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113334    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="alertmanager"
Oct 30 11:20:23.113454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113393    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="alertmanager"
Oct 30 11:20:23.113454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.113402    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c756993-cd82-48eb-837e-219a7e989822" containerName="config-reloader"
Oct 30 11:20:23.118092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.118074    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.141110 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.138960    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 11:20:23.141110 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.139035    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 11:20:23.141110 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.139066    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:20:23.141110 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.139337    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 11:20:23.141110 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.139574    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-4cc5x\""
Oct 30 11:20:23.141110 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.141058    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:20:23.173868 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.173842    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:20:23.177610 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.177591    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.205414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.205384    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.205524 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.205445    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.205524 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.205514    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.205627 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.205542    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.205682 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.205646    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.205736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.205704    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9lcc\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-kube-api-access-v9lcc\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.205816 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.205800    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.211262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.211239    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:20:23.238822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.238791    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 11:20:23.238942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.238836    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 11:20:23.238942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.238848    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-w4zq4\""
Oct 30 11:20:23.238942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.238834    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 11:20:23.238942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.238869    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:20:23.238942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.238876    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 11:20:23.238942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.238909    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 11:20:23.270837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.270819    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 11:20:23.270950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.270936    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 11:20:23.271096 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.271082    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 11:20:23.306852 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.306830    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftsnj\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-kube-api-access-ftsnj\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.306939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.306862    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.306939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.306886    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.306946    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.306971    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.307014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.306988    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307007    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307209 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307073    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307250 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307225    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-tls-assets\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307286 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307268    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307349 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307328    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.307419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307401    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9lcc\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-kube-api-access-v9lcc\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.307483 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307445    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307537 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307476    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307537 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307512    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.307648 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307559    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307648 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307613    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.307768 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307653    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.307768 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307679    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/050550e8-d339-4011-98e6-777bcf2e6edc-config-out\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.307953 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.307930    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.309672 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.309642    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.309852 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.309835    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.309937 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.309917    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.309976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.309927    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.310041 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.310001    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.332671 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.332649    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9lcc\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-kube-api-access-v9lcc\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.408198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408153    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408283 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408200    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408283 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408220    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408283 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408242    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408283 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408271    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/050550e8-d339-4011-98e6-777bcf2e6edc-config-out\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408479 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408288    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftsnj\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-kube-api-access-ftsnj\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408479 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408304    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408479 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408432    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408479 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408463    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408719 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408494    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408719 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408526    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.408719 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.408563    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-tls-assets\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.409144 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.409116    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.409310 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.409281    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.409722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.409612    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.411059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.410993    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/050550e8-d339-4011-98e6-777bcf2e6edc-config-out\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.411167 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.411107    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.411289 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.411261    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.411354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.411302    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-tls-assets\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.411561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.411543    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.411860 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.411842    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.412122 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.412102    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.412533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.412512    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.433146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.433126    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftsnj\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-kube-api-access-ftsnj\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:23.435941 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.435926    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:23.751086 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.751061    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:20:23.752608 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:20:23.752583    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2426a254_6287_4e40_bb10_775f3a250648.slice/crio-4bf7298e9012b0227888b95e2c7f8ecf5c5616a135aa4ea435f6beab18b2660c WatchSource:0}: Error finding container 4bf7298e9012b0227888b95e2c7f8ecf5c5616a135aa4ea435f6beab18b2660c: Status 404 returned error can't find the container with id 4bf7298e9012b0227888b95e2c7f8ecf5c5616a135aa4ea435f6beab18b2660c
Oct 30 11:20:23.850344 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:23.850312    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerStarted","Data":"4bf7298e9012b0227888b95e2c7f8ecf5c5616a135aa4ea435f6beab18b2660c"}
Oct 30 11:20:24.753118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:24.753078    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c756993-cd82-48eb-837e-219a7e989822" path="/var/lib/kubelet/pods/8c756993-cd82-48eb-837e-219a7e989822/volumes"
Oct 30 11:20:24.857290 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:24.857251    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerStarted","Data":"3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a"}
Oct 30 11:20:27.241133 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.241098    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.272746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.272719    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.342166 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.342141    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.342405 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.342385    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.357895 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.357868    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") DevicePath \"csi-bc5b0eea9a26be0763bd2234369df1d0cb0bca5610c96b98228e00775a514ae6\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.459805 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.459772    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/7342da0537aa528cc1545d2018ac3fd0382dfe4f670067e58cf8ba34bb891174/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.488895 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.488875    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.697175 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.697153    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:27.873623 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.873552    4849 generic.go:356] "Generic (PLEG): container finished" podID="2426a254-6287-4e40-bb10-775f3a250648" containerID="3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a" exitCode=0
Oct 30 11:20:27.873623 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:27.873614    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerDied","Data":"3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a"}
Oct 30 11:20:28.018718 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:28.018695    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:20:28.020554 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:20:28.020530    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050550e8_d339_4011_98e6_777bcf2e6edc.slice/crio-051e61b0f42f9c0313a77a572a7991ff9eadb9f8e94864a25f7659ea5eecdce7 WatchSource:0}: Error finding container 051e61b0f42f9c0313a77a572a7991ff9eadb9f8e94864a25f7659ea5eecdce7: Status 404 returned error can't find the container with id 051e61b0f42f9c0313a77a572a7991ff9eadb9f8e94864a25f7659ea5eecdce7
Oct 30 11:20:28.879818 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:28.879784    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerStarted","Data":"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"}
Oct 30 11:20:28.880244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:28.879851    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerStarted","Data":"051e61b0f42f9c0313a77a572a7991ff9eadb9f8e94864a25f7659ea5eecdce7"}
Oct 30 11:20:28.882072 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:28.882046    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerStarted","Data":"97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311"}
Oct 30 11:20:28.882191 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:28.882099    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerStarted","Data":"47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be"}
Oct 30 11:20:28.882322 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:28.882305    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:28.999155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:28.999107    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podStartSLOduration=2.513226859 podStartE2EDuration="5.999094796s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.485867937s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:20:28.997246757 +0000 UTC m=+2126.377696196" watchObservedRunningTime="2026-10-30 11:20:28.999094796 +0000 UTC m=+2126.379544235"
Oct 30 11:20:31.900957 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:31.900923    4849 generic.go:356] "Generic (PLEG): container finished" podID="050550e8-d339-4011-98e6-777bcf2e6edc" containerID="90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78" exitCode=0
Oct 30 11:20:31.901461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:31.901011    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerDied","Data":"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"}
Oct 30 11:20:32.908876 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:32.908834    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerStarted","Data":"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"}
Oct 30 11:20:32.909259 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:32.908911    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerStarted","Data":"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"}
Oct 30 11:20:32.909259 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:32.908937    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerStarted","Data":"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"}
Oct 30 11:20:33.080148 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:33.080093    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podStartSLOduration=6.694464541 podStartE2EDuration="10.080077012s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.385612471s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:20:33.065881502 +0000 UTC m=+2130.446330941" watchObservedRunningTime="2026-10-30 11:20:33.080077012 +0000 UTC m=+2130.460526454"
Oct 30 11:20:34.889333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:34.889303    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:20:37.698151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:37.698119    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:39.431205 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:20:39.431168    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:20:42.698102 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:42.698053    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:42.703673 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:42.703644    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:20:42.773986 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:42.773946    4849 pod_container_manager_linux.go:218] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod3e34af22-5522-43d8-b7cd-5f2d4067d51d"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod3e34af22-5522-43d8-b7cd-5f2d4067d51d] : Timed out while waiting for systemd to remove kubepods-besteffort-pod3e34af22_5522_43d8_b7cd_5f2d4067d51d.slice"
Oct 30 11:20:42.774136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:42.773985    4849 kubelet_pods.go:2797] "Failed to destroy orphaned pod cgroup" podUID="3e34af22-5522-43d8-b7cd-5f2d4067d51d" err="failed to delete cgroup paths for [kubepods besteffort pod3e34af22-5522-43d8-b7cd-5f2d4067d51d] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod3e34af22-5522-43d8-b7cd-5f2d4067d51d] : Timed out while waiting for systemd to remove kubepods-besteffort-pod3e34af22_5522_43d8_b7cd_5f2d4067d51d.slice"
Oct 30 11:20:42.968158 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:20:42.968076    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:21:09.431772 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:09.431729    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:21:27.753329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.753297    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:21:27.753885 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.753511    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="alertmanager" containerID="cri-o://47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be" gracePeriod=120
Oct 30 11:21:27.753885 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.753569    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="config-reloader" containerID="cri-o://97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311" gracePeriod=120
Oct 30 11:21:27.774138 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.774106    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:21:27.774446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.774400    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="thanos-sidecar" containerID="cri-o://4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73" gracePeriod=600
Oct 30 11:21:27.774542 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.774451    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="config-reloader" containerID="cri-o://51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac" gracePeriod=600
Oct 30 11:21:27.774605 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.774399    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="prometheus" containerID="cri-o://a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11" gracePeriod=600
Oct 30 11:21:27.964376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:27.964345    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="prometheus" probeResult="failure" output="Get \"https://10.132.0.95:9090/-/ready\": dial tcp 10.132.0.95:9090: connect: connection refused"
Oct 30 11:21:28.121266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.121243    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:21:28.209139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209110    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-config\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209154    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-tls-assets\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-tls-assets\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209194    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209223    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-kube-api-access-ftsnj\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-kube-api-access-ftsnj\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209252    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-thanos-prometheus-http-client-file\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209299    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209370    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209397    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209437    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209465    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/050550e8-d339-4011-98e6-777bcf2e6edc-config-out\" (UniqueName: \"kubernetes.io/empty-dir/050550e8-d339-4011-98e6-777bcf2e6edc-config-out\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209767 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209588    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209767 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209629    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-secret-prometheus-web-tls-ca\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.209767 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.209678    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"050550e8-d339-4011-98e6-777bcf2e6edc\" (UID: \"050550e8-d339-4011-98e6-777bcf2e6edc\") "
Oct 30 11:21:28.210262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.210219    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-2" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:21:28.210615 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.210587    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-1" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:21:28.211359 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.211074    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-0" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:21:28.212235 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.212207    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.213189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.213051    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-config" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.213189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.213064    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-kube-api-access-ftsnj" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "kube-api-access-ftsnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:21:28.213189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.213127    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-tls-assets" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:21:28.214173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.214145    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.214173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.214162    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/050550e8-d339-4011-98e6-777bcf2e6edc-config-out" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:21:28.214626 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.214602    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-thanos-prometheus-http-client-file" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.214909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.214886    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-secret-prometheus-web-tls-ca" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.215887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.215857    4849 generic.go:356] "Generic (PLEG): container finished" podID="050550e8-d339-4011-98e6-777bcf2e6edc" containerID="4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73" exitCode=0
Oct 30 11:21:28.215887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.215887    4849 generic.go:356] "Generic (PLEG): container finished" podID="050550e8-d339-4011-98e6-777bcf2e6edc" containerID="51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac" exitCode=0
Oct 30 11:21:28.216090 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.215898    4849 generic.go:356] "Generic (PLEG): container finished" podID="050550e8-d339-4011-98e6-777bcf2e6edc" containerID="a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11" exitCode=0
Oct 30 11:21:28.216090 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.215925    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:21:28.216090 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.215941    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerDied","Data":"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"}
Oct 30 11:21:28.216090 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.216036    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerDied","Data":"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"}
Oct 30 11:21:28.216318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.216101    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerDied","Data":"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"}
Oct 30 11:21:28.216318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.216143    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"050550e8-d339-4011-98e6-777bcf2e6edc","Type":"ContainerDied","Data":"051e61b0f42f9c0313a77a572a7991ff9eadb9f8e94864a25f7659ea5eecdce7"}
Oct 30 11:21:28.216318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.216187    4849 scope.go:122] "RemoveContainer" containerID="4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"
Oct 30 11:21:28.216318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.216281    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "pvc-c799cc48-125a-421c-8fc6-8a4d017a7653". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:21:28.218634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.218613    4849 generic.go:356] "Generic (PLEG): container finished" podID="2426a254-6287-4e40-bb10-775f3a250648" containerID="97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311" exitCode=0
Oct 30 11:21:28.218744 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.218689    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerDied","Data":"97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311"}
Oct 30 11:21:28.226012 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.225981    4849 scope.go:122] "RemoveContainer" containerID="51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"
Oct 30 11:21:28.228469 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.228445    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config" pod "050550e8-d339-4011-98e6-777bcf2e6edc" (UID: "050550e8-d339-4011-98e6-777bcf2e6edc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.235644 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.235620    4849 scope.go:122] "RemoveContainer" containerID="a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"
Oct 30 11:21:28.244168 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.244151    4849 scope.go:122] "RemoveContainer" containerID="90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"
Oct 30 11:21:28.251667 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.251651    4849 scope.go:122] "RemoveContainer" containerID="4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"
Oct 30 11:21:28.251925 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:28.251903    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73\": container with ID starting with 4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73 not found: ID does not exist" containerID="4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"
Oct 30 11:21:28.251969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.251932    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"} err="failed to get container status \"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73\": rpc error: code = NotFound desc = could not find container \"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73\": container with ID starting with 4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73 not found: ID does not exist"
Oct 30 11:21:28.251969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.251946    4849 scope.go:122] "RemoveContainer" containerID="51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"
Oct 30 11:21:28.252180 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:28.252165    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac\": container with ID starting with 51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac not found: ID does not exist" containerID="51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"
Oct 30 11:21:28.252222 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.252183    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"} err="failed to get container status \"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac\": rpc error: code = NotFound desc = could not find container \"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac\": container with ID starting with 51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac not found: ID does not exist"
Oct 30 11:21:28.252222 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.252196    4849 scope.go:122] "RemoveContainer" containerID="a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"
Oct 30 11:21:28.252453 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:28.252427    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11\": container with ID starting with a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11 not found: ID does not exist" containerID="a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"
Oct 30 11:21:28.252548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.252458    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"} err="failed to get container status \"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11\": rpc error: code = NotFound desc = could not find container \"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11\": container with ID starting with a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11 not found: ID does not exist"
Oct 30 11:21:28.252548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.252489    4849 scope.go:122] "RemoveContainer" containerID="90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"
Oct 30 11:21:28.252749 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:28.252731    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78\": container with ID starting with 90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78 not found: ID does not exist" containerID="90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"
Oct 30 11:21:28.252804 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.252755    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"} err="failed to get container status \"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78\": rpc error: code = NotFound desc = could not find container \"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78\": container with ID starting with 90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78 not found: ID does not exist"
Oct 30 11:21:28.252804 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.252769    4849 scope.go:122] "RemoveContainer" containerID="4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"
Oct 30 11:21:28.253118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253095    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"} err="failed to get container status \"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73\": rpc error: code = NotFound desc = could not find container \"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73\": container with ID starting with 4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73 not found: ID does not exist"
Oct 30 11:21:28.253118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253116    4849 scope.go:122] "RemoveContainer" containerID="51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"
Oct 30 11:21:28.253362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253344    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"} err="failed to get container status \"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac\": rpc error: code = NotFound desc = could not find container \"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac\": container with ID starting with 51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac not found: ID does not exist"
Oct 30 11:21:28.253413 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253370    4849 scope.go:122] "RemoveContainer" containerID="a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"
Oct 30 11:21:28.253635 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253618    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"} err="failed to get container status \"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11\": rpc error: code = NotFound desc = could not find container \"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11\": container with ID starting with a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11 not found: ID does not exist"
Oct 30 11:21:28.253678 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253636    4849 scope.go:122] "RemoveContainer" containerID="90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"
Oct 30 11:21:28.253901 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253881    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"} err="failed to get container status \"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78\": rpc error: code = NotFound desc = could not find container \"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78\": container with ID starting with 90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78 not found: ID does not exist"
Oct 30 11:21:28.253947 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.253912    4849 scope.go:122] "RemoveContainer" containerID="4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"
Oct 30 11:21:28.254198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.254178    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73"} err="failed to get container status \"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73\": rpc error: code = NotFound desc = could not find container \"4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73\": container with ID starting with 4791d01fc502b7e581ed7b8015d9be1366443b04d6cc8ad9fb46baf32349ee73 not found: ID does not exist"
Oct 30 11:21:28.254198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.254197    4849 scope.go:122] "RemoveContainer" containerID="51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"
Oct 30 11:21:28.254440 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.254422    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac"} err="failed to get container status \"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac\": rpc error: code = NotFound desc = could not find container \"51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac\": container with ID starting with 51197f95443ba710d807da3fddf98e85acefe7f82ca937d39b55e89d3853aeac not found: ID does not exist"
Oct 30 11:21:28.254491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.254441    4849 scope.go:122] "RemoveContainer" containerID="a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"
Oct 30 11:21:28.254673 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.254656    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11"} err="failed to get container status \"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11\": rpc error: code = NotFound desc = could not find container \"a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11\": container with ID starting with a4a184ebd02b7685363b979f0985af39fe4637335f018c688157f1f1af201d11 not found: ID does not exist"
Oct 30 11:21:28.254754 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.254674    4849 scope.go:122] "RemoveContainer" containerID="90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"
Oct 30 11:21:28.254898 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.254883    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78"} err="failed to get container status \"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78\": rpc error: code = NotFound desc = could not find container \"90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78\": container with ID starting with 90c4c50f95dc7c2c39f6bf320bda24d46d4095f454e05d01a6dfd3b6ecedac78 not found: ID does not exist"
Oct 30 11:21:28.311317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311285    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311315    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311326    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311337    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311353    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftsnj\" (UniqueName: \"kubernetes.io/projected/050550e8-d339-4011-98e6-777bcf2e6edc-kube-api-access-ftsnj\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311365    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311375    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311384    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311393    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311403    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/050550e8-d339-4011-98e6-777bcf2e6edc-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311411    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/050550e8-d339-4011-98e6-777bcf2e6edc-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.311444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311442    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:21:28.311733 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.311452    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/050550e8-d339-4011-98e6-777bcf2e6edc-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.325478 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.325462    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-c799cc48-125a-421c-8fc6-8a4d017a7653" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:21:28.359567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.359537    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:21:28.359801 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.359776    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="71ffb0d0-4afa-448e-9164-56a4d6ecefcf" containerName="otc-container" containerID="cri-o://95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe" gracePeriod=30
Oct 30 11:21:28.412223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.412192    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-bc5b0eea9a26be0763bd2234369df1d0cb0bca5610c96b98228e00775a514ae6\""
Oct 30 11:21:28.570360 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.570271    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:21:28.574333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.574306    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:21:28.605701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.605679    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:21:28.714409 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.714378    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-kube-api-access-rhzsz\" (UniqueName: \"kubernetes.io/projected/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-kube-api-access-rhzsz\") pod \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") "
Oct 30 11:21:28.714554 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.714419    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-otc-internal\" (UniqueName: \"kubernetes.io/configmap/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-otc-internal\") pod \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") "
Oct 30 11:21:28.714621 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.714599    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-tls-certs\") pod \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") "
Oct 30 11:21:28.714656 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.714641    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-data-science-collector-ta-client-cert\") pod \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") "
Oct 30 11:21:28.714720 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.714705    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-monitoring-tls-certs\") pod \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\" (UID: \"71ffb0d0-4afa-448e-9164-56a4d6ecefcf\") "
Oct 30 11:21:28.714785 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.714766    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-otc-internal" pod "71ffb0d0-4afa-448e-9164-56a4d6ecefcf" (UID: "71ffb0d0-4afa-448e-9164-56a4d6ecefcf"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:21:28.714950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.714935    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.716535 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.716504    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-tls-certs" pod "71ffb0d0-4afa-448e-9164-56a4d6ecefcf" (UID: "71ffb0d0-4afa-448e-9164-56a4d6ecefcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.716636 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.716557    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-kube-api-access-rhzsz" pod "71ffb0d0-4afa-448e-9164-56a4d6ecefcf" (UID: "71ffb0d0-4afa-448e-9164-56a4d6ecefcf"). InnerVolumeSpecName "kube-api-access-rhzsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:21:28.716636 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.716587    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-data-science-collector-ta-client-cert" pod "71ffb0d0-4afa-448e-9164-56a4d6ecefcf" (UID: "71ffb0d0-4afa-448e-9164-56a4d6ecefcf"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.716636 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.716618    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-monitoring-tls-certs" pod "71ffb0d0-4afa-448e-9164-56a4d6ecefcf" (UID: "71ffb0d0-4afa-448e-9164-56a4d6ecefcf"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:28.755260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.755227    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" path="/var/lib/kubelet/pods/050550e8-d339-4011-98e6-777bcf2e6edc/volumes"
Oct 30 11:21:28.815946 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.815923    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.815946 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.815947    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.816092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.815956    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:28.816092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:28.815966    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhzsz\" (UniqueName: \"kubernetes.io/projected/71ffb0d0-4afa-448e-9164-56a4d6ecefcf-kube-api-access-rhzsz\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:29.224459 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.224356    4849 generic.go:356] "Generic (PLEG): container finished" podID="71ffb0d0-4afa-448e-9164-56a4d6ecefcf" containerID="95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe" exitCode=0
Oct 30 11:21:29.224459 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.224404    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:21:29.224459 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.224439    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"71ffb0d0-4afa-448e-9164-56a4d6ecefcf","Type":"ContainerDied","Data":"95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe"}
Oct 30 11:21:29.224711 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.224490    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"71ffb0d0-4afa-448e-9164-56a4d6ecefcf","Type":"ContainerDied","Data":"c0de4a1b9b3b0e32258374277701b041581977994ab739f21348f4f06a364f62"}
Oct 30 11:21:29.224711 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.224532    4849 scope.go:122] "RemoveContainer" containerID="95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe"
Oct 30 11:21:29.232921 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.232903    4849 scope.go:122] "RemoveContainer" containerID="95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe"
Oct 30 11:21:29.233211 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:29.233190    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe\": container with ID starting with 95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe not found: ID does not exist" containerID="95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe"
Oct 30 11:21:29.233294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.233223    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe"} err="failed to get container status \"95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe\": rpc error: code = NotFound desc = could not find container \"95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe\": container with ID starting with 95c8d1045a7533d3fa278cc613a1106e8470400159bd7d37e5e443463f5fc4fe not found: ID does not exist"
Oct 30 11:21:29.282220 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.282183    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:21:29.287258 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.287230    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:21:29.888273 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:29.888236    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.94:9093/-/ready\": dial tcp 10.132.0.94:9093: connect: connection refused"
Oct 30 11:21:30.753828 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:30.753790    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71ffb0d0-4afa-448e-9164-56a4d6ecefcf" path="/var/lib/kubelet/pods/71ffb0d0-4afa-448e-9164-56a4d6ecefcf/volumes"
Oct 30 11:21:34.887947 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:34.887898    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.94:9093/-/ready\": dial tcp 10.132.0.94:9093: connect: connection refused"
Oct 30 11:21:38.001380 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.001356    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:21:38.083386 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083352    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-alertmanager-data-science-monitoringstack-db\") pod \"2426a254-6287-4e40-bb10-775f3a250648\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") "
Oct 30 11:21:38.083386 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083387    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-config-out\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-config-out\") pod \"2426a254-6287-4e40-bb10-775f3a250648\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") "
Oct 30 11:21:38.083582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083426    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-cluster-tls-config\") pod \"2426a254-6287-4e40-bb10-775f3a250648\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") "
Oct 30 11:21:38.083582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083452    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-config-volume\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-config-volume\") pod \"2426a254-6287-4e40-bb10-775f3a250648\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") "
Oct 30 11:21:38.083582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083468    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-web-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-web-config\") pod \"2426a254-6287-4e40-bb10-775f3a250648\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") "
Oct 30 11:21:38.083582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083495    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-kube-api-access-v9lcc\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-kube-api-access-v9lcc\") pod \"2426a254-6287-4e40-bb10-775f3a250648\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") "
Oct 30 11:21:38.083582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083524    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-tls-assets\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-tls-assets\") pod \"2426a254-6287-4e40-bb10-775f3a250648\" (UID: \"2426a254-6287-4e40-bb10-775f3a250648\") "
Oct 30 11:21:38.083849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083719    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-alertmanager-data-science-monitoringstack-db" pod "2426a254-6287-4e40-bb10-775f3a250648" (UID: "2426a254-6287-4e40-bb10-775f3a250648"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:21:38.083849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.083832    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:38.085691 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.085666    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-config-volume" pod "2426a254-6287-4e40-bb10-775f3a250648" (UID: "2426a254-6287-4e40-bb10-775f3a250648"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:38.086070 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.086013    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-kube-api-access-v9lcc" pod "2426a254-6287-4e40-bb10-775f3a250648" (UID: "2426a254-6287-4e40-bb10-775f3a250648"). InnerVolumeSpecName "kube-api-access-v9lcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:21:38.086182 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.086129    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-tls-assets" pod "2426a254-6287-4e40-bb10-775f3a250648" (UID: "2426a254-6287-4e40-bb10-775f3a250648"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:21:38.086308 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.086284    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-config-out" pod "2426a254-6287-4e40-bb10-775f3a250648" (UID: "2426a254-6287-4e40-bb10-775f3a250648"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:21:38.089840 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.089812    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-cluster-tls-config" pod "2426a254-6287-4e40-bb10-775f3a250648" (UID: "2426a254-6287-4e40-bb10-775f3a250648"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:38.098477 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.098442    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-web-config" pod "2426a254-6287-4e40-bb10-775f3a250648" (UID: "2426a254-6287-4e40-bb10-775f3a250648"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:21:38.185135 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.185113    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2426a254-6287-4e40-bb10-775f3a250648-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:38.185135 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.185135    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:38.185260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.185145    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:38.185260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.185154    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2426a254-6287-4e40-bb10-775f3a250648-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:38.185260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.185162    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9lcc\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-kube-api-access-v9lcc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:38.185260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.185170    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2426a254-6287-4e40-bb10-775f3a250648-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:21:38.279531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.279505    4849 generic.go:356] "Generic (PLEG): container finished" podID="2426a254-6287-4e40-bb10-775f3a250648" containerID="47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be" exitCode=0
Oct 30 11:21:38.279638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.279553    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerDied","Data":"47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be"}
Oct 30 11:21:38.279638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.279589    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:21:38.279638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.279613    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"2426a254-6287-4e40-bb10-775f3a250648","Type":"ContainerDied","Data":"4bf7298e9012b0227888b95e2c7f8ecf5c5616a135aa4ea435f6beab18b2660c"}
Oct 30 11:21:38.279638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.279638    4849 scope.go:122] "RemoveContainer" containerID="97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311"
Oct 30 11:21:38.288382 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.288364    4849 scope.go:122] "RemoveContainer" containerID="47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be"
Oct 30 11:21:38.297512 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.297495    4849 scope.go:122] "RemoveContainer" containerID="3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a"
Oct 30 11:21:38.304853 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.304838    4849 scope.go:122] "RemoveContainer" containerID="97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311"
Oct 30 11:21:38.305148 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:38.305131    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311\": container with ID starting with 97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311 not found: ID does not exist" containerID="97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311"
Oct 30 11:21:38.305199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.305155    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311"} err="failed to get container status \"97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311\": rpc error: code = NotFound desc = could not find container \"97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311\": container with ID starting with 97f27d199be492a0bdebd0a2f29d2ec8e9ebb12db73b8077552d1ab699085311 not found: ID does not exist"
Oct 30 11:21:38.305199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.305172    4849 scope.go:122] "RemoveContainer" containerID="47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be"
Oct 30 11:21:38.305412 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:38.305392    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be\": container with ID starting with 47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be not found: ID does not exist" containerID="47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be"
Oct 30 11:21:38.305461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.305423    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be"} err="failed to get container status \"47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be\": rpc error: code = NotFound desc = could not find container \"47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be\": container with ID starting with 47c3bbb2b959bbe74df3f0aa3474f321a096166458f237c493bd72d40d0c74be not found: ID does not exist"
Oct 30 11:21:38.305461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.305453    4849 scope.go:122] "RemoveContainer" containerID="3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a"
Oct 30 11:21:38.305701 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:38.305681    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a\": container with ID starting with 3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a not found: ID does not exist" containerID="3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a"
Oct 30 11:21:38.305788 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.305704    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a"} err="failed to get container status \"3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a\": rpc error: code = NotFound desc = could not find container \"3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a\": container with ID starting with 3a2ed969899787d9438d61b8b4097d6051aa4d3a4a06f0eccff0ca78322bd47a not found: ID does not exist"
Oct 30 11:21:38.335441 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.335382    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:21:38.338920 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.338903    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:21:38.754193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:38.754129    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2426a254-6287-4e40-bb10-775f3a250648" path="/var/lib/kubelet/pods/2426a254-6287-4e40-bb10-775f3a250648/volumes"
Oct 30 11:21:39.431415 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:21:39.431378    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:21:51.503275 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503237    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"]
Oct 30 11:21:51.503753 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503731    4849 cpu_manager.go:414] "containerMap: removing container" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="init-config-reloader"
Oct 30 11:21:51.503807 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503757    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="init-config-reloader"
Oct 30 11:21:51.503807 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503772    4849 cpu_manager.go:414] "containerMap: removing container" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="config-reloader"
Oct 30 11:21:51.503807 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503779    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="config-reloader"
Oct 30 11:21:51.503807 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503802    4849 cpu_manager.go:414] "containerMap: removing container" podUID="71ffb0d0-4afa-448e-9164-56a4d6ecefcf" containerName="otc-container"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503810    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71ffb0d0-4afa-448e-9164-56a4d6ecefcf" containerName="otc-container"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503826    4849 cpu_manager.go:414] "containerMap: removing container" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="alertmanager"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503831    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="alertmanager"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503847    4849 cpu_manager.go:414] "containerMap: removing container" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="prometheus"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503851    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="prometheus"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503860    4849 cpu_manager.go:414] "containerMap: removing container" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="init-config-reloader"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503865    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="init-config-reloader"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503877    4849 cpu_manager.go:414] "containerMap: removing container" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="config-reloader"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503882    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="config-reloader"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503887    4849 cpu_manager.go:414] "containerMap: removing container" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="thanos-sidecar"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503891    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="thanos-sidecar"
Oct 30 11:21:51.503943 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503946    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="alertmanager"
Oct 30 11:21:51.504318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503955    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="prometheus"
Oct 30 11:21:51.504318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503964    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="2426a254-6287-4e40-bb10-775f3a250648" containerName="config-reloader"
Oct 30 11:21:51.504318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503970    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="thanos-sidecar"
Oct 30 11:21:51.504318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503976    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="050550e8-d339-4011-98e6-777bcf2e6edc" containerName="config-reloader"
Oct 30 11:21:51.504318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.503982    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="71ffb0d0-4afa-448e-9164-56a4d6ecefcf" containerName="otc-container"
Oct 30 11:21:51.508449 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.508430    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.528075 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.528049    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"]
Oct 30 11:21:51.584340 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.584308    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-query-frontend-http\""
Oct 30 11:21:51.584340 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.584308    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-query-frontend-grpc\""
Oct 30 11:21:51.584493 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.584467    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-dockercfg-4pmvt\""
Oct 30 11:21:51.609397 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.609367    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:21:51.609509 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.609480    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ca-bundle\""
Oct 30 11:21:51.609557 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.609534    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:21:51.609634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.609619    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-config\""
Oct 30 11:21:51.683961 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.683932    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.684102 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.683988    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-grpc\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.684156 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.684121    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-http\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.684193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.684160    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-config\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.684241 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.684223    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59svq\" (UniqueName: \"kubernetes.io/projected/23f37a36-405c-477a-943a-2126b21f7d23-kube-api-access-59svq\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.784775 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.784715    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-config\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.784775 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.784749    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59svq\" (UniqueName: \"kubernetes.io/projected/23f37a36-405c-477a-943a-2126b21f7d23-kube-api-access-59svq\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.784775 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.784773    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.784948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.784812    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-grpc\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.784948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.784860    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-http\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.785654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.785615    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.785770 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.785636    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-config\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.787256 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.787239    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-http\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.787331 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.787313    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-grpc\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.814241 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.814221    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59svq\" (UniqueName: \"kubernetes.io/projected/23f37a36-405c-477a-943a-2126b21f7d23-kube-api-access-59svq\") pod \"data-science-lokistack-query-frontend-f746fc66d-fl67f\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:51.820937 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:51.820915    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:52.121513 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:52.121485    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"]
Oct 30 11:21:52.124650 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:21:52.124614    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23f37a36_405c_477a_943a_2126b21f7d23.slice/crio-51f2d4122c79ae8dd036e22e9f1c062a2af88ac7b9070ddd7de2829e6085e2ed WatchSource:0}: Error finding container 51f2d4122c79ae8dd036e22e9f1c062a2af88ac7b9070ddd7de2829e6085e2ed: Status 404 returned error can't find the container with id 51f2d4122c79ae8dd036e22e9f1c062a2af88ac7b9070ddd7de2829e6085e2ed
Oct 30 11:21:52.126809 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:52.126787    4849 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 11:21:52.352069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:52.352018    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f" event={"ID":"23f37a36-405c-477a-943a-2126b21f7d23","Type":"ContainerStarted","Data":"51f2d4122c79ae8dd036e22e9f1c062a2af88ac7b9070ddd7de2829e6085e2ed"}
Oct 30 11:21:54.299880 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.299828    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:21:54.304118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.304096    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.320874 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.320809    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"test-loki-shared-secret\""
Oct 30 11:21:54.321084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.320955    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-http\""
Oct 30 11:21:54.334397 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.334373    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-grpc\""
Oct 30 11:21:54.336562 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.336542    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:21:54.409214 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.409180    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.409359 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.409245    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-test-loki-shared-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.409359 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.409332    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.409478 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.409382    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.409536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.409490    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.409536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.409510    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8fzm\" (UniqueName: \"kubernetes.io/projected/321675d9-08c8-478d-94e6-e574a875fd3b-kube-api-access-t8fzm\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.457295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.457269    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-compactor-0"]
Oct 30 11:21:54.460883 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.460866    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.480209 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.480186    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-compactor-grpc\""
Oct 30 11:21:54.482258 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.482237    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-compactor-http\""
Oct 30 11:21:54.482877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.482856    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-compactor-0"]
Oct 30 11:21:54.509925 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.509901    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.510065 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.509955    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.510065 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.509979    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8fzm\" (UniqueName: \"kubernetes.io/projected/321675d9-08c8-478d-94e6-e574a875fd3b-kube-api-access-t8fzm\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.510199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.510161    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.510257 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.510219    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-test-loki-shared-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.510319 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.510254    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.510879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.510834    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.510879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.510871    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.513422 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.513386    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.513931 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.513719    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-test-loki-shared-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.520555 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.520527    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.534277 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.534258    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8fzm\" (UniqueName: \"kubernetes.io/projected/321675d9-08c8-478d-94e6-e574a875fd3b-kube-api-access-t8fzm\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:54.611255 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.611230    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-config\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.611384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.611268    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-test-loki-shared-secret\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.611384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.611298    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-http\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.611489 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.611436    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.611538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.611489    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c6qr\" (UniqueName: \"kubernetes.io/projected/11ca0059-d7ab-4908-a08b-268ea6832483-kube-api-access-5c6qr\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.611538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.611526    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-grpc\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.712236 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.712203    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c6qr\" (UniqueName: \"kubernetes.io/projected/11ca0059-d7ab-4908-a08b-268ea6832483-kube-api-access-5c6qr\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.712377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.712251    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-grpc\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.712377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.712352    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-config\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.712485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.712387    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-test-loki-shared-secret\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.712485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.712418    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-http\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.712485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.712455    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.713251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.713205    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.713360 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.713329    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-config\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.715398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.715376    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-http\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.715678 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.715654    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-test-loki-shared-secret\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.716173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.716152    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-grpc\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:54.737591 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:54.737568    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c6qr\" (UniqueName: \"kubernetes.io/projected/11ca0059-d7ab-4908-a08b-268ea6832483-kube-api-access-5c6qr\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:55.370340 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:55.370257    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f" event={"ID":"23f37a36-405c-477a-943a-2126b21f7d23","Type":"ContainerStarted","Data":"957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a"}
Oct 30 11:21:55.370722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:55.370409    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:21:55.415541 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:55.415474    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f" podStartSLOduration=1.453734041 podStartE2EDuration="4.41546241s" totalImagesPullingTime="2.961728369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:21:55.414784582 +0000 UTC m=+2212.795234021" watchObservedRunningTime="2026-10-30 11:21:55.41546241 +0000 UTC m=+2212.795911848"
Oct 30 11:21:59.147060 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.147011    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.147568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.147069    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.147568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.147085    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.192987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.192958    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.192987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.192974    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.193159 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.192983    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.248329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.248305    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.248430 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.248344    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.248430 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.248366    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.248501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.248439    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.248501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.248455    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.248501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.248472    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.264094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.264070    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") DevicePath \"csi-3a06fd0f3b2a24302a943d2e07f8c74fd3599fdd9788d3a5d7c7a38ff823c062\"" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.264171 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.264131    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") DevicePath \"csi-ec75d9b0b91142f6b6afd54340144a51026b4e387d2663fc875e3386d5d871bf\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.264227 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.264132    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") DevicePath \"csi-cde36451543a15adc4b8ffa9b9955623c11fd95bdb37e3b1a03a871bcc3f2ab0\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.493163 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.493136    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/1c19039a2608c8bacc8ebf206c5def5c3154a220ec38c0ab0294eb81856ec5cd/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.515748 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.515724    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/7df80f80ed0e4c658d21d8a82ac94fa2db4558ef72248dc2d4a0162a6c11c695/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.520272 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.520252    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:21:59.539939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.539918    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"data-science-lokistack-compactor-0\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") " pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.573794 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.573775    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:21:59.874665 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:21:59.874622    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-compactor-0"]
Oct 30 11:21:59.877328 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:21:59.877305    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ca0059_d7ab_4908_a08b_268ea6832483.slice/crio-e9df5cb26b651528f10cc8ace7450aa1f0923db014517013d537378e7d49f05d WatchSource:0}: Error finding container e9df5cb26b651528f10cc8ace7450aa1f0923db014517013d537378e7d49f05d: Status 404 returned error can't find the container with id e9df5cb26b651528f10cc8ace7450aa1f0923db014517013d537378e7d49f05d
Oct 30 11:22:00.401941 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:00.401905    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0" event={"ID":"11ca0059-d7ab-4908-a08b-268ea6832483","Type":"ContainerStarted","Data":"1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28"}
Oct 30 11:22:00.402451 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:00.401960    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0" event={"ID":"11ca0059-d7ab-4908-a08b-268ea6832483","Type":"ContainerStarted","Data":"e9df5cb26b651528f10cc8ace7450aa1f0923db014517013d537378e7d49f05d"}
Oct 30 11:22:00.402451 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:00.402006    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:22:00.449838 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:00.449790    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0" podStartSLOduration=9.449779621 podStartE2EDuration="9.449779621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:22:00.447361965 +0000 UTC m=+2217.827811416" watchObservedRunningTime="2026-10-30 11:22:00.449779621 +0000 UTC m=+2217.830229060"
Oct 30 11:22:06.587276 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:06.587244    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/f1ba4cd0a8ebf37ed4c6577964876a5661084ff0c33c7df67c3521fff1e49e3c/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:22:06.613449 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:06.613413    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:22:06.917165 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:06.917138    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:22:07.254186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:07.254157    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:22:07.255854 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:22:07.255824    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321675d9_08c8_478d_94e6_e574a875fd3b.slice/crio-79aca9a5e3ccf224fb6c7c9c348950be9520fbf8ae847a03d8865c87314adec0 WatchSource:0}: Error finding container 79aca9a5e3ccf224fb6c7c9c348950be9520fbf8ae847a03d8865c87314adec0: Status 404 returned error can't find the container with id 79aca9a5e3ccf224fb6c7c9c348950be9520fbf8ae847a03d8865c87314adec0
Oct 30 11:22:07.441287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:07.441252    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"321675d9-08c8-478d-94e6-e574a875fd3b","Type":"ContainerStarted","Data":"6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613"}
Oct 30 11:22:07.441461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:07.441309    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"321675d9-08c8-478d-94e6-e574a875fd3b","Type":"ContainerStarted","Data":"79aca9a5e3ccf224fb6c7c9c348950be9520fbf8ae847a03d8865c87314adec0"}
Oct 30 11:22:07.441461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:07.441343    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:22:07.489956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:07.489907    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podStartSLOduration=16.489894279 podStartE2EDuration="16.489894279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:22:07.488828883 +0000 UTC m=+2224.869278321" watchObservedRunningTime="2026-10-30 11:22:07.489894279 +0000 UTC m=+2224.870343717"
Oct 30 11:22:09.432750 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:22:09.432714    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:22:16.382066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:16.382002    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:22:21.414138 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:21.414103    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:22:26.580399 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.580363    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:22:26.584227 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.584209    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.616583 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.616560    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-index-gateway-http\""
Oct 30 11:22:26.616748 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.616699    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-index-gateway-grpc\""
Oct 30 11:22:26.618729 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.618702    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:22:26.761073 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.761041    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-grpc\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.761194 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.761093    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.761194 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.761178    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhmqq\" (UniqueName: \"kubernetes.io/projected/db29cdcd-5b3a-4560-88eb-150f5baa767c-kube-api-access-mhmqq\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.761284 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.761211    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-test-loki-shared-secret\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.761284 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.761235    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-config\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.761284 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.761251    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-http\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.862481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.862412    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.862481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.862461    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhmqq\" (UniqueName: \"kubernetes.io/projected/db29cdcd-5b3a-4560-88eb-150f5baa767c-kube-api-access-mhmqq\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.862481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.862484    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-test-loki-shared-secret\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.862680 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.862593    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-config\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.862680 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.862633    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-http\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.862785 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.862685    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-grpc\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.863344 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.863315    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.863466 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.863394    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-config\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.865206 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.865182    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-grpc\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.865507 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.865487    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-http\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.865744 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.865725    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-test-loki-shared-secret\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:26.887285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:26.887262    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhmqq\" (UniqueName: \"kubernetes.io/projected/db29cdcd-5b3a-4560-88eb-150f5baa767c-kube-api-access-mhmqq\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:28.452913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:28.452873    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:22:29.783724 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:29.783692    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:29.815731 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:29.815700    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:29.884928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:29.884900    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:29.885068 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:29.884999    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:29.900959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:29.900934    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") DevicePath \"csi-599188236b712b00d635aa56bbdfd3e8788ebbbf3062cdbf2fb2dbf91bd4ee6b\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:31.070622 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:31.070594    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5b6238a2954ef95199bf19d6b2a92fd4c269274ee1250fa34a57dae345cc9d9f/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:31.097223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:31.097198    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:31.397478 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:31.397447    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:31.709388 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:31.709362    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:22:31.711008 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:22:31.710982    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb29cdcd_5b3a_4560_88eb_150f5baa767c.slice/crio-7c4505b369b59e10d62b90ac0db71c7483288ab31c79cd6300732874c3bbd7fa WatchSource:0}: Error finding container 7c4505b369b59e10d62b90ac0db71c7483288ab31c79cd6300732874c3bbd7fa: Status 404 returned error can't find the container with id 7c4505b369b59e10d62b90ac0db71c7483288ab31c79cd6300732874c3bbd7fa
Oct 30 11:22:32.573203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:32.573160    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"db29cdcd-5b3a-4560-88eb-150f5baa767c","Type":"ContainerStarted","Data":"907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca"}
Oct 30 11:22:32.573743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:32.573230    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"db29cdcd-5b3a-4560-88eb-150f5baa767c","Type":"ContainerStarted","Data":"7c4505b369b59e10d62b90ac0db71c7483288ab31c79cd6300732874c3bbd7fa"}
Oct 30 11:22:32.573743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:32.573405    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:32.636643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:32.636595    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" podStartSLOduration=9.63658386 podStartE2EDuration="9.63658386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:22:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:22:32.634366826 +0000 UTC m=+2250.014816275" watchObservedRunningTime="2026-10-30 11:22:32.63658386 +0000 UTC m=+2250.017033298"
Oct 30 11:22:38.452121 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:38.452064    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:22:39.429663 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:22:39.429622    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:22:48.452081 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:48.452012    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:22:53.583446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:53.583409    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:22:58.451116 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:22:58.451087    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:23:09.432125 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:23:09.432084    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:23:39.429940 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:23:39.429861    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:24:09.432654 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:24:09.432604    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:24:39.429977 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:24:39.429939    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:25:03.017195 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:03.017099    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:25:03.028811 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:03.028790    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:25:09.432337 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:09.432299    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:25:30.881327 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:30.881292    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"]
Oct 30 11:25:30.884914 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:30.884892    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:30.905246 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:30.905218    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"]
Oct 30 11:25:30.916288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:30.916259    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-dockercfg-tk7xf\""
Oct 30 11:25:30.916388 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:30.916369    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-6a6878d3\""
Oct 30 11:25:31.001913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.001888    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78abe738-b00b-4b2f-956c-602a8e28294e-otc-internal\") pod \"data-science-usage-logs-collector-697dbdbd58-9hrfx\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:31.002049 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.002037    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjkcs\" (UniqueName: \"kubernetes.io/projected/78abe738-b00b-4b2f-956c-602a8e28294e-kube-api-access-tjkcs\") pod \"data-science-usage-logs-collector-697dbdbd58-9hrfx\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:31.102580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.102557    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjkcs\" (UniqueName: \"kubernetes.io/projected/78abe738-b00b-4b2f-956c-602a8e28294e-kube-api-access-tjkcs\") pod \"data-science-usage-logs-collector-697dbdbd58-9hrfx\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:31.102670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.102596    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78abe738-b00b-4b2f-956c-602a8e28294e-otc-internal\") pod \"data-science-usage-logs-collector-697dbdbd58-9hrfx\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:31.103252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.103233    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78abe738-b00b-4b2f-956c-602a8e28294e-otc-internal\") pod \"data-science-usage-logs-collector-697dbdbd58-9hrfx\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:31.127158 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.127130    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjkcs\" (UniqueName: \"kubernetes.io/projected/78abe738-b00b-4b2f-956c-602a8e28294e-kube-api-access-tjkcs\") pod \"data-science-usage-logs-collector-697dbdbd58-9hrfx\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:31.199789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.199738    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:31.498356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.498332    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"]
Oct 30 11:25:31.500260 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:25:31.500235    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78abe738_b00b_4b2f_956c_602a8e28294e.slice/crio-3145c652def46d81379677ca9d1bf3298c79e0bbac33e979f4b47f5317de73df WatchSource:0}: Error finding container 3145c652def46d81379677ca9d1bf3298c79e0bbac33e979f4b47f5317de73df: Status 404 returned error can't find the container with id 3145c652def46d81379677ca9d1bf3298c79e0bbac33e979f4b47f5317de73df
Oct 30 11:25:31.502767 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.502741    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:25:31.502827 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:31.502794    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:25:32.484735 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:32.484700    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx" event={"ID":"78abe738-b00b-4b2f-956c-602a8e28294e","Type":"ContainerStarted","Data":"6c0c7d7ba62ff26ff0abd283bb321e1c7c7447eab4da0bb6d04dbb55f96434d2"}
Oct 30 11:25:32.485244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:32.484761    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx" event={"ID":"78abe738-b00b-4b2f-956c-602a8e28294e","Type":"ContainerStarted","Data":"3145c652def46d81379677ca9d1bf3298c79e0bbac33e979f4b47f5317de73df"}
Oct 30 11:25:32.529326 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:32.529278    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx" podStartSLOduration=2.529264652 podStartE2EDuration="2.529264652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:25:32.527044909 +0000 UTC m=+2429.907494346" watchObservedRunningTime="2026-10-30 11:25:32.529264652 +0000 UTC m=+2429.909714148"
Oct 30 11:25:34.430308 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.430267    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-compactor-0"]
Oct 30 11:25:34.430768 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.430512    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0" podUID="11ca0059-d7ab-4908-a08b-268ea6832483" containerName="loki-compactor" containerID="cri-o://1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28" gracePeriod=30
Oct 30 11:25:34.443149 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.443111    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"]
Oct 30 11:25:34.446640 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.446622    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.462732 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.462710    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"]
Oct 30 11:25:34.466552 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.466530    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-querier-grpc\""
Oct 30 11:25:34.466660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.466554    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"test-loki-lifecycle-secret\""
Oct 30 11:25:34.466660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.466594    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-querier-http\""
Oct 30 11:25:34.478219 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.478199    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:25:34.478390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.478372    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" podUID="db29cdcd-5b3a-4560-88eb-150f5baa767c" containerName="loki-index-gateway" containerID="cri-o://907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca" gracePeriod=30
Oct 30 11:25:34.635224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.635193    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.635384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.635236    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.635384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.635365    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.635468 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.635404    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-config\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.635468 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.635429    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-test-loki-lifecycle-secret\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.635548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.635471    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88lz5\" (UniqueName: \"kubernetes.io/projected/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-kube-api-access-88lz5\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.736582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.736553    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.736702 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.736594    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-config\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.736702 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.736690    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-test-loki-lifecycle-secret\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.736833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.736707    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88lz5\" (UniqueName: \"kubernetes.io/projected/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-kube-api-access-88lz5\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.736833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.736766    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.736833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.736793    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.737356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.737329    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.737489 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.737452    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-config\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.739240 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.739219    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.739368 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.739351    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.739829 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.739803    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-test-loki-lifecycle-secret\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.744736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.744717    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:34.761551 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.761532    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88lz5\" (UniqueName: \"kubernetes.io/projected/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-kube-api-access-88lz5\") pod \"data-science-lokistack-querier-56ffbd5fdc-7dclw\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:34.938232 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938205    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-test-loki-shared-secret\") pod \"db29cdcd-5b3a-4560-88eb-150f5baa767c\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") "
Oct 30 11:25:34.938394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938253    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-grpc\") pod \"db29cdcd-5b3a-4560-88eb-150f5baa767c\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") "
Oct 30 11:25:34.938394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938277    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-ca-bundle\") pod \"db29cdcd-5b3a-4560-88eb-150f5baa767c\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") "
Oct 30 11:25:34.938394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938295    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-http\") pod \"db29cdcd-5b3a-4560-88eb-150f5baa767c\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") "
Oct 30 11:25:34.938394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938334    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db29cdcd-5b3a-4560-88eb-150f5baa767c-kube-api-access-mhmqq\" (UniqueName: \"kubernetes.io/projected/db29cdcd-5b3a-4560-88eb-150f5baa767c-kube-api-access-mhmqq\") pod \"db29cdcd-5b3a-4560-88eb-150f5baa767c\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") "
Oct 30 11:25:34.938600 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938426    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-config\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-config\") pod \"db29cdcd-5b3a-4560-88eb-150f5baa767c\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") "
Oct 30 11:25:34.938600 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938530    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"db29cdcd-5b3a-4560-88eb-150f5baa767c\" (UID: \"db29cdcd-5b3a-4560-88eb-150f5baa767c\") "
Oct 30 11:25:34.939269 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.938859    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-ca-bundle" pod "db29cdcd-5b3a-4560-88eb-150f5baa767c" (UID: "db29cdcd-5b3a-4560-88eb-150f5baa767c"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:25:34.939269 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.939080    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-config" pod "db29cdcd-5b3a-4560-88eb-150f5baa767c" (UID: "db29cdcd-5b3a-4560-88eb-150f5baa767c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:25:34.940896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.940870    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-test-loki-shared-secret" pod "db29cdcd-5b3a-4560-88eb-150f5baa767c" (UID: "db29cdcd-5b3a-4560-88eb-150f5baa767c"). InnerVolumeSpecName "test-loki-shared-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:25:34.941017 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.940927    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-http" pod "db29cdcd-5b3a-4560-88eb-150f5baa767c" (UID: "db29cdcd-5b3a-4560-88eb-150f5baa767c"). InnerVolumeSpecName "data-science-lokistack-index-gateway-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:25:34.941092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.941071    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db29cdcd-5b3a-4560-88eb-150f5baa767c-kube-api-access-mhmqq" pod "db29cdcd-5b3a-4560-88eb-150f5baa767c" (UID: "db29cdcd-5b3a-4560-88eb-150f5baa767c"). InnerVolumeSpecName "kube-api-access-mhmqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:25:34.941281 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.941254    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-grpc" pod "db29cdcd-5b3a-4560-88eb-150f5baa767c" (UID: "db29cdcd-5b3a-4560-88eb-150f5baa767c"). InnerVolumeSpecName "data-science-lokistack-index-gateway-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:25:34.941754 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:34.941736    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015" pod "db29cdcd-5b3a-4560-88eb-150f5baa767c" (UID: "db29cdcd-5b3a-4560-88eb-150f5baa767c"). InnerVolumeSpecName "pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:25:35.039426 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.039379    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:25:35.039426 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.039400    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-test-loki-shared-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.039426 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.039410    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.039426 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.039420    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.039426 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.039430    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/db29cdcd-5b3a-4560-88eb-150f5baa767c-data-science-lokistack-index-gateway-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.039729 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.039438    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhmqq\" (UniqueName: \"kubernetes.io/projected/db29cdcd-5b3a-4560-88eb-150f5baa767c-kube-api-access-mhmqq\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.039729 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.039447    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db29cdcd-5b3a-4560-88eb-150f5baa767c-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.051428 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.051411    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:25:35.059461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.059440    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:35.140690 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.140663    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-599188236b712b00d635aa56bbdfd3e8788ebbbf3062cdbf2fb2dbf91bd4ee6b\""
Oct 30 11:25:35.266676 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.266652    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:25:35.342387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342354    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-ca-bundle\") pod \"11ca0059-d7ab-4908-a08b-268ea6832483\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") "
Oct 30 11:25:35.342563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342428    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-http\") pod \"11ca0059-d7ab-4908-a08b-268ea6832483\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") "
Oct 30 11:25:35.342563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342448    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11ca0059-d7ab-4908-a08b-268ea6832483-kube-api-access-5c6qr\" (UniqueName: \"kubernetes.io/projected/11ca0059-d7ab-4908-a08b-268ea6832483-kube-api-access-5c6qr\") pod \"11ca0059-d7ab-4908-a08b-268ea6832483\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") "
Oct 30 11:25:35.342563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342541    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") pod \"11ca0059-d7ab-4908-a08b-268ea6832483\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") "
Oct 30 11:25:35.342748 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342570    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-config\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-config\") pod \"11ca0059-d7ab-4908-a08b-268ea6832483\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") "
Oct 30 11:25:35.342748 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342589    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-test-loki-shared-secret\") pod \"11ca0059-d7ab-4908-a08b-268ea6832483\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") "
Oct 30 11:25:35.342748 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342642    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-grpc\") pod \"11ca0059-d7ab-4908-a08b-268ea6832483\" (UID: \"11ca0059-d7ab-4908-a08b-268ea6832483\") "
Oct 30 11:25:35.342901 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342771    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-ca-bundle" pod "11ca0059-d7ab-4908-a08b-268ea6832483" (UID: "11ca0059-d7ab-4908-a08b-268ea6832483"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:25:35.342954 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.342929    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.343411 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.343384    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-config" pod "11ca0059-d7ab-4908-a08b-268ea6832483" (UID: "11ca0059-d7ab-4908-a08b-268ea6832483"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:25:35.345206 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.345122    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-http" pod "11ca0059-d7ab-4908-a08b-268ea6832483" (UID: "11ca0059-d7ab-4908-a08b-268ea6832483"). InnerVolumeSpecName "data-science-lokistack-compactor-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:25:35.345329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.345211    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-grpc" pod "11ca0059-d7ab-4908-a08b-268ea6832483" (UID: "11ca0059-d7ab-4908-a08b-268ea6832483"). InnerVolumeSpecName "data-science-lokistack-compactor-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:25:35.345329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.345296    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ca0059-d7ab-4908-a08b-268ea6832483-kube-api-access-5c6qr" pod "11ca0059-d7ab-4908-a08b-268ea6832483" (UID: "11ca0059-d7ab-4908-a08b-268ea6832483"). InnerVolumeSpecName "kube-api-access-5c6qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:25:35.345485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.345466    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-test-loki-shared-secret" pod "11ca0059-d7ab-4908-a08b-268ea6832483" (UID: "11ca0059-d7ab-4908-a08b-268ea6832483"). InnerVolumeSpecName "test-loki-shared-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:25:35.345933 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.345912    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895" pod "11ca0059-d7ab-4908-a08b-268ea6832483" (UID: "11ca0059-d7ab-4908-a08b-268ea6832483"). InnerVolumeSpecName "pvc-7d33c086-8411-4c5e-bbb7-fd829bece632". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:25:35.358782 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.358721    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"]
Oct 30 11:25:35.360936 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:25:35.360916    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39eee566_f2de_4c32_87b0_1ad6b34e4a2c.slice/crio-71f66e1de7dfe05ad53d93b26ee30ed7c7ff1d79a2b7bc3a771466c718d34d1b WatchSource:0}: Error finding container 71f66e1de7dfe05ad53d93b26ee30ed7c7ff1d79a2b7bc3a771466c718d34d1b: Status 404 returned error can't find the container with id 71f66e1de7dfe05ad53d93b26ee30ed7c7ff1d79a2b7bc3a771466c718d34d1b
Oct 30 11:25:35.443350 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.443326    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.443716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.443356    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c6qr\" (UniqueName: \"kubernetes.io/projected/11ca0059-d7ab-4908-a08b-268ea6832483-kube-api-access-5c6qr\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.443716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.443390    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:25:35.443716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.443403    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11ca0059-d7ab-4908-a08b-268ea6832483-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.443716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.443412    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-test-loki-shared-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.443716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.443421    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/11ca0059-d7ab-4908-a08b-268ea6832483-data-science-lokistack-compactor-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:35.453822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.453801    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-7d33c086-8411-4c5e-bbb7-fd829bece632" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:25:35.506076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.505979    4849 generic.go:356] "Generic (PLEG): container finished" podID="11ca0059-d7ab-4908-a08b-268ea6832483" containerID="1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28" exitCode=0
Oct 30 11:25:35.506199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.506119    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0" event={"ID":"11ca0059-d7ab-4908-a08b-268ea6832483","Type":"ContainerDied","Data":"1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28"}
Oct 30 11:25:35.506263 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.506204    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0" event={"ID":"11ca0059-d7ab-4908-a08b-268ea6832483","Type":"ContainerDied","Data":"e9df5cb26b651528f10cc8ace7450aa1f0923db014517013d537378e7d49f05d"}
Oct 30 11:25:35.506263 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.506232    4849 scope.go:122] "RemoveContainer" containerID="1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28"
Oct 30 11:25:35.506364 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.506277    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-compactor-0"
Oct 30 11:25:35.510208 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.509766    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw" event={"ID":"39eee566-f2de-4c32-87b0-1ad6b34e4a2c","Type":"ContainerStarted","Data":"78f12af48586580b13cc12b3291586b12453859aeb4d736a94e0c40e601fb191"}
Oct 30 11:25:35.510208 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.509835    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw" event={"ID":"39eee566-f2de-4c32-87b0-1ad6b34e4a2c","Type":"ContainerStarted","Data":"71f66e1de7dfe05ad53d93b26ee30ed7c7ff1d79a2b7bc3a771466c718d34d1b"}
Oct 30 11:25:35.510208 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.509913    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:25:35.512301 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.512276    4849 generic.go:356] "Generic (PLEG): container finished" podID="db29cdcd-5b3a-4560-88eb-150f5baa767c" containerID="907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca" exitCode=0
Oct 30 11:25:35.512731 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.512633    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.513257 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.512649    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"db29cdcd-5b3a-4560-88eb-150f5baa767c","Type":"ContainerDied","Data":"907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca"}
Oct 30 11:25:35.513354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.513276    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"db29cdcd-5b3a-4560-88eb-150f5baa767c","Type":"ContainerDied","Data":"7c4505b369b59e10d62b90ac0db71c7483288ab31c79cd6300732874c3bbd7fa"}
Oct 30 11:25:35.523233 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.523146    4849 scope.go:122] "RemoveContainer" containerID="1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28"
Oct 30 11:25:35.523557 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:35.523430    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28\": container with ID starting with 1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28 not found: ID does not exist" containerID="1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28"
Oct 30 11:25:35.523557 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.523462    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28"} err="failed to get container status \"1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28\": rpc error: code = NotFound desc = could not find container \"1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28\": container with ID starting with 1233f344189a4c1568e36670b61f86da5c5978ac3ab8c419212935bd1e6c9c28 not found: ID does not exist"
Oct 30 11:25:35.523557 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.523485    4849 scope.go:122] "RemoveContainer" containerID="907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca"
Oct 30 11:25:35.533816 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.533793    4849 scope.go:122] "RemoveContainer" containerID="907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca"
Oct 30 11:25:35.534122 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:35.534098    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca\": container with ID starting with 907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca not found: ID does not exist" containerID="907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca"
Oct 30 11:25:35.534199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.534128    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca"} err="failed to get container status \"907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca\": rpc error: code = NotFound desc = could not find container \"907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca\": container with ID starting with 907f32ecc3562e9d7ebe6d53d66331b499838b1d600d5c34cf7cebf89c8ee9ca not found: ID does not exist"
Oct 30 11:25:35.543822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.543801    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-7d33c086-8411-4c5e-bbb7-fd829bece632\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019deea708189b895\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-3a06fd0f3b2a24302a943d2e07f8c74fd3599fdd9788d3a5d7c7a38ff823c062\""
Oct 30 11:25:35.570752 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.570710    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw" podStartSLOduration=1.57070028 podStartE2EDuration="1.57070028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:25:35.569771272 +0000 UTC m=+2432.950220713" watchObservedRunningTime="2026-10-30 11:25:35.57070028 +0000 UTC m=+2432.951149710"
Oct 30 11:25:35.610256 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.610187    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-compactor-0"]
Oct 30 11:25:35.613833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.613806    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-compactor-0"]
Oct 30 11:25:35.649789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.649765    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:25:35.655436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.655414    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:25:35.692076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.692049    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:25:35.692937 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.692914    4849 cpu_manager.go:414] "containerMap: removing container" podUID="11ca0059-d7ab-4908-a08b-268ea6832483" containerName="loki-compactor"
Oct 30 11:25:35.693034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.692940    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11ca0059-d7ab-4908-a08b-268ea6832483" containerName="loki-compactor"
Oct 30 11:25:35.693034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.692993    4849 cpu_manager.go:414] "containerMap: removing container" podUID="db29cdcd-5b3a-4560-88eb-150f5baa767c" containerName="loki-index-gateway"
Oct 30 11:25:35.693034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.693001    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db29cdcd-5b3a-4560-88eb-150f5baa767c" containerName="loki-index-gateway"
Oct 30 11:25:35.693260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.693245    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="11ca0059-d7ab-4908-a08b-268ea6832483" containerName="loki-compactor"
Oct 30 11:25:35.693315 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.693274    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="db29cdcd-5b3a-4560-88eb-150f5baa767c" containerName="loki-index-gateway"
Oct 30 11:25:35.699981 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.699956    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.715179 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.715153    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:25:35.773089 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.773068    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-index-gateway-grpc\""
Oct 30 11:25:35.773089 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.773068    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-index-gateway-http\""
Oct 30 11:25:35.845856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.845827    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-config\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.845963 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.845873    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.846008 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.845968    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-grpc\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.846087 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.846045    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmkcz\" (UniqueName: \"kubernetes.io/projected/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-kube-api-access-fmkcz\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.846192 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.846175    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-http\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.846256 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.846206    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-test-loki-lifecycle-secret\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.947244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.947187    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-http\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.947244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.947216    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-test-loki-lifecycle-secret\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.947405 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.947387    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-config\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.947469 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.947425    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.947469 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.947464    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-grpc\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.947578 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.947507    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmkcz\" (UniqueName: \"kubernetes.io/projected/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-kube-api-access-fmkcz\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.948291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.948270    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-config\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.948408 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.948270    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.950123 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.950103    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-grpc\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.950229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.950213    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-http\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.950630 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.950607    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-test-loki-lifecycle-secret\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:35.971619 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:35.971589    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmkcz\" (UniqueName: \"kubernetes.io/projected/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-kube-api-access-fmkcz\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:36.498224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.498198    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"]
Oct 30 11:25:36.502929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.502913    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:36.527231 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.527205    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"]
Oct 30 11:25:36.653513 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.653483    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcgx4\" (UniqueName: \"kubernetes.io/projected/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-kube-api-access-pcgx4\") pod \"data-science-usage-logs-collector-5896c48b7-b5v4z\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:36.653643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.653538    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal\") pod \"data-science-usage-logs-collector-5896c48b7-b5v4z\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:36.753887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.753830    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11ca0059-d7ab-4908-a08b-268ea6832483" path="/var/lib/kubelet/pods/11ca0059-d7ab-4908-a08b-268ea6832483/volumes"
Oct 30 11:25:36.754286 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.754271    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db29cdcd-5b3a-4560-88eb-150f5baa767c" path="/var/lib/kubelet/pods/db29cdcd-5b3a-4560-88eb-150f5baa767c/volumes"
Oct 30 11:25:36.754749 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.754732    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgx4\" (UniqueName: \"kubernetes.io/projected/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-kube-api-access-pcgx4\") pod \"data-science-usage-logs-collector-5896c48b7-b5v4z\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:36.754799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.754757    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal\") pod \"data-science-usage-logs-collector-5896c48b7-b5v4z\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:36.755324 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.755305    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal\") pod \"data-science-usage-logs-collector-5896c48b7-b5v4z\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:36.779338 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.779315    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcgx4\" (UniqueName: \"kubernetes.io/projected/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-kube-api-access-pcgx4\") pod \"data-science-usage-logs-collector-5896c48b7-b5v4z\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:36.816245 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:36.816226    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:37.119801 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.119778    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"]
Oct 30 11:25:37.121850 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:25:37.121824    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3b7d93_4d8a_4be8_b711_e5c9db5980a5.slice/crio-11fcf7853d7b5bf8ebbe9cf0388e6a82b2d3cc3e0d492ea07fbdcb741c8d08e1 WatchSource:0}: Error finding container 11fcf7853d7b5bf8ebbe9cf0388e6a82b2d3cc3e0d492ea07fbdcb741c8d08e1: Status 404 returned error can't find the container with id 11fcf7853d7b5bf8ebbe9cf0388e6a82b2d3cc3e0d492ea07fbdcb741c8d08e1
Oct 30 11:25:37.124142 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.124110    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:25:37.124244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.124175    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:25:37.331538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.331506    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"]
Oct 30 11:25:37.331764 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.331739    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx" podUID="78abe738-b00b-4b2f-956c-602a8e28294e" containerName="otc-container" containerID="cri-o://6c0c7d7ba62ff26ff0abd283bb321e1c7c7447eab4da0bb6d04dbb55f96434d2" gracePeriod=30
Oct 30 11:25:37.338859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.338830    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"]
Oct 30 11:25:37.528762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.528727    4849 generic.go:356] "Generic (PLEG): container finished" podID="78abe738-b00b-4b2f-956c-602a8e28294e" containerID="6c0c7d7ba62ff26ff0abd283bb321e1c7c7447eab4da0bb6d04dbb55f96434d2" exitCode=0
Oct 30 11:25:37.529166 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.528800    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx" event={"ID":"78abe738-b00b-4b2f-956c-602a8e28294e","Type":"ContainerDied","Data":"6c0c7d7ba62ff26ff0abd283bb321e1c7c7447eab4da0bb6d04dbb55f96434d2"}
Oct 30 11:25:37.530755 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.530729    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z" event={"ID":"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5","Type":"ContainerStarted","Data":"aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f"}
Oct 30 11:25:37.530872 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.530781    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z" event={"ID":"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5","Type":"ContainerStarted","Data":"11fcf7853d7b5bf8ebbe9cf0388e6a82b2d3cc3e0d492ea07fbdcb741c8d08e1"}
Oct 30 11:25:37.530935 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.530911    4849 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z" secret="" err="secret \"data-science-usage-logs-collector-dockercfg-tk7xf\" not found"
Oct 30 11:25:37.569286 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.569269    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:37.575774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.575734    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z" podStartSLOduration=1.5757241130000001 podStartE2EDuration="1.575724113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:25:37.573973898 +0000 UTC m=+2434.954423337" watchObservedRunningTime="2026-10-30 11:25:37.575724113 +0000 UTC m=+2434.956173551"
Oct 30 11:25:37.662355 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.662332    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78abe738-b00b-4b2f-956c-602a8e28294e-kube-api-access-tjkcs\" (UniqueName: \"kubernetes.io/projected/78abe738-b00b-4b2f-956c-602a8e28294e-kube-api-access-tjkcs\") pod \"78abe738-b00b-4b2f-956c-602a8e28294e\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") "
Oct 30 11:25:37.662454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.662398    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78abe738-b00b-4b2f-956c-602a8e28294e-otc-internal\" (UniqueName: \"kubernetes.io/configmap/78abe738-b00b-4b2f-956c-602a8e28294e-otc-internal\") pod \"78abe738-b00b-4b2f-956c-602a8e28294e\" (UID: \"78abe738-b00b-4b2f-956c-602a8e28294e\") "
Oct 30 11:25:37.662625 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:37.662612    4849 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-usage-logs-collector-6a6878d3: configmap "data-science-usage-logs-collector-6a6878d3" not found
Oct 30 11:25:37.662702 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:37.662662    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal podName:5a3b7d93-4d8a-4be8-b711-e5c9db5980a5 nodeName:}" failed. No retries permitted until 2026-10-30 11:25:38.1626473 +0000 UTC m=+2435.543096730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "otc-internal" (UniqueName: "kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal") pod "data-science-usage-logs-collector-5896c48b7-b5v4z" (UID: "5a3b7d93-4d8a-4be8-b711-e5c9db5980a5") : configmap "data-science-usage-logs-collector-6a6878d3" not found
Oct 30 11:25:37.662702 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.662684    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78abe738-b00b-4b2f-956c-602a8e28294e-otc-internal" pod "78abe738-b00b-4b2f-956c-602a8e28294e" (UID: "78abe738-b00b-4b2f-956c-602a8e28294e"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:25:37.664419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.664400    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78abe738-b00b-4b2f-956c-602a8e28294e-kube-api-access-tjkcs" pod "78abe738-b00b-4b2f-956c-602a8e28294e" (UID: "78abe738-b00b-4b2f-956c-602a8e28294e"). InnerVolumeSpecName "kube-api-access-tjkcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:25:37.763833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.763784    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/78abe738-b00b-4b2f-956c-602a8e28294e-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:37.763833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:37.763804    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjkcs\" (UniqueName: \"kubernetes.io/projected/78abe738-b00b-4b2f-956c-602a8e28294e-kube-api-access-tjkcs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:38.167620 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:38.167585    4849 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-usage-logs-collector-6a6878d3: configmap "data-science-usage-logs-collector-6a6878d3" not found
Oct 30 11:25:38.167760 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:38.167666    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal podName:5a3b7d93-4d8a-4be8-b711-e5c9db5980a5 nodeName:}" failed. No retries permitted until 2026-10-30 11:25:39.167645356 +0000 UTC m=+2436.548094778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "otc-internal" (UniqueName: "kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal") pod "data-science-usage-logs-collector-5896c48b7-b5v4z" (UID: "5a3b7d93-4d8a-4be8-b711-e5c9db5980a5") : configmap "data-science-usage-logs-collector-6a6878d3" not found
Oct 30 11:25:38.537620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.537553    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx" event={"ID":"78abe738-b00b-4b2f-956c-602a8e28294e","Type":"ContainerDied","Data":"3145c652def46d81379677ca9d1bf3298c79e0bbac33e979f4b47f5317de73df"}
Oct 30 11:25:38.537620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.537576    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"
Oct 30 11:25:38.537620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.537609    4849 scope.go:122] "RemoveContainer" containerID="6c0c7d7ba62ff26ff0abd283bb321e1c7c7447eab4da0bb6d04dbb55f96434d2"
Oct 30 11:25:38.538199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.537638    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z" podUID="5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" containerName="otc-container" containerID="cri-o://aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f" gracePeriod=30
Oct 30 11:25:38.636457 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.636423    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"]
Oct 30 11:25:38.640720 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.640700    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-697dbdbd58-9hrfx"]
Oct 30 11:25:38.755115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.754996    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78abe738-b00b-4b2f-956c-602a8e28294e" path="/var/lib/kubelet/pods/78abe738-b00b-4b2f-956c-602a8e28294e/volumes"
Oct 30 11:25:38.807787 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.807765    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:38.974235 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.974208    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-kube-api-access-pcgx4\" (UniqueName: \"kubernetes.io/projected/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-kube-api-access-pcgx4\") pod \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") "
Oct 30 11:25:38.974365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.974279    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal\" (UniqueName: \"kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal\") pod \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\" (UID: \"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5\") "
Oct 30 11:25:38.974659 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.974634    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal" pod "5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" (UID: "5a3b7d93-4d8a-4be8-b711-e5c9db5980a5"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:25:38.975986 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:38.975965    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-kube-api-access-pcgx4" pod "5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" (UID: "5a3b7d93-4d8a-4be8-b711-e5c9db5980a5"). InnerVolumeSpecName "kube-api-access-pcgx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:25:39.075663 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.075641    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:39.075663 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.075664    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcgx4\" (UniqueName: \"kubernetes.io/projected/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5-kube-api-access-pcgx4\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:25:39.430811 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:39.430740    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:25:39.545619 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.545590    4849 generic.go:356] "Generic (PLEG): container finished" podID="5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" containerID="aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f" exitCode=0
Oct 30 11:25:39.545998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.545635    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"
Oct 30 11:25:39.545998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.545671    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z" event={"ID":"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5","Type":"ContainerDied","Data":"aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f"}
Oct 30 11:25:39.545998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.545719    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z" event={"ID":"5a3b7d93-4d8a-4be8-b711-e5c9db5980a5","Type":"ContainerDied","Data":"11fcf7853d7b5bf8ebbe9cf0388e6a82b2d3cc3e0d492ea07fbdcb741c8d08e1"}
Oct 30 11:25:39.545998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.545751    4849 scope.go:122] "RemoveContainer" containerID="aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f"
Oct 30 11:25:39.554573 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.554556    4849 scope.go:122] "RemoveContainer" containerID="aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f"
Oct 30 11:25:39.554864 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:25:39.554842    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f\": container with ID starting with aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f not found: ID does not exist" containerID="aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f"
Oct 30 11:25:39.554933 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.554870    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f"} err="failed to get container status \"aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f\": rpc error: code = NotFound desc = could not find container \"aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f\": container with ID starting with aab8444e4e1333cff6c18d70e35754b44c3c5e911f9bd1eb9c743b3658ec1c3f not found: ID does not exist"
Oct 30 11:25:39.602541 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.602515    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"]
Oct 30 11:25:39.615684 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:39.615662    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-b5v4z"]
Oct 30 11:25:40.753832 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:40.753783    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" path="/var/lib/kubelet/pods/5a3b7d93-4d8a-4be8-b711-e5c9db5980a5/volumes"
Oct 30 11:25:43.614240 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:43.614207    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:43.647007 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:43.646978    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:43.715096 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:43.715067    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:43.715228 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:43.715207    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:43.730707 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:43.730684    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") DevicePath \"csi-599188236b712b00d635aa56bbdfd3e8788ebbbf3062cdbf2fb2dbf91bd4ee6b\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:43.835696 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:43.835675    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5b6238a2954ef95199bf19d6b2a92fd4c269274ee1250fa34a57dae345cc9d9f/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:43.862684 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:43.862662    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"data-science-lokistack-index-gateway-1\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") " pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:44.115696 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:44.115671    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:44.419962 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:44.419936    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:25:44.422040 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:25:44.421991    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae495a4_3ba8_4348_bb5d_1c5b9adb8bcd.slice/crio-d39c038486b2334e3a74f80dbc03243b6dc2169055cf6c5d0e8d0327af20daa0 WatchSource:0}: Error finding container d39c038486b2334e3a74f80dbc03243b6dc2169055cf6c5d0e8d0327af20daa0: Status 404 returned error can't find the container with id d39c038486b2334e3a74f80dbc03243b6dc2169055cf6c5d0e8d0327af20daa0
Oct 30 11:25:44.574306 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:44.574266    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd","Type":"ContainerStarted","Data":"4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908"}
Oct 30 11:25:44.574463 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:44.574319    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd","Type":"ContainerStarted","Data":"d39c038486b2334e3a74f80dbc03243b6dc2169055cf6c5d0e8d0327af20daa0"}
Oct 30 11:25:44.574463 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:44.574350    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:25:44.621148 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:44.621096    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" podStartSLOduration=9.621083215 podStartE2EDuration="9.621083215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:25:44.619790075 +0000 UTC m=+2442.000239518" watchObservedRunningTime="2026-10-30 11:25:44.621083215 +0000 UTC m=+2442.001532655"
Oct 30 11:25:56.526386 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:25:56.526349    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:26:05.585781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:26:05.585750    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:26:09.434823 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:26:09.434781    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:26:39.430377 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:26:39.430297    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:27:09.433970 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:27:09.433930    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:27:18.723489 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:18.723446    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:27:18.724181 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:18.723685    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" containerName="loki-ingester" containerID="cri-o://6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613" gracePeriod=600
Oct 30 11:27:19.876934 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.876908    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:19.957424 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957367    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957424 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957419    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ca-bundle\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957441    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-test-loki-shared-secret\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957465    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-http\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957529    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957602    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-config\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-config\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957861 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957643    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-grpc\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957861 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957674    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/321675d9-08c8-478d-94e6-e574a875fd3b-kube-api-access-t8fzm\" (UniqueName: \"kubernetes.io/projected/321675d9-08c8-478d-94e6-e574a875fd3b-kube-api-access-t8fzm\") pod \"321675d9-08c8-478d-94e6-e574a875fd3b\" (UID: \"321675d9-08c8-478d-94e6-e574a875fd3b\") "
Oct 30 11:27:19.957975 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.957946    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ca-bundle" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:27:19.958212 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.958157    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-config" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:27:19.959905 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.959873    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-http" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "data-science-lokistack-ingester-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:27:19.960171 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.960133    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-test-loki-shared-secret" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "test-loki-shared-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:27:19.960171 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.960146    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321675d9-08c8-478d-94e6-e574a875fd3b-kube-api-access-t8fzm" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "kube-api-access-t8fzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:27:19.960300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.960256    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-grpc" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "data-science-lokistack-ingester-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:27:19.960780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.960755    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:27:19.960896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:19.960881    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3" pod "321675d9-08c8-478d-94e6-e574a875fd3b" (UID: "321675d9-08c8-478d-94e6-e574a875fd3b"). InnerVolumeSpecName "pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:27:20.058649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058627    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:27:20.058649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058647    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:27:20.058777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058658    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-shared-secret\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-test-loki-shared-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:27:20.058777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058668    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:27:20.058777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058684    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:27:20.058777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058694    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/321675d9-08c8-478d-94e6-e574a875fd3b-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:27:20.058777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058707    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/321675d9-08c8-478d-94e6-e574a875fd3b-data-science-lokistack-ingester-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:27:20.058777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.058720    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8fzm\" (UniqueName: \"kubernetes.io/projected/321675d9-08c8-478d-94e6-e574a875fd3b-kube-api-access-t8fzm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:27:20.070074 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.070050    4849 generic.go:356] "Generic (PLEG): container finished" podID="321675d9-08c8-478d-94e6-e574a875fd3b" containerID="6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613" exitCode=0
Oct 30 11:27:20.070159 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.070096    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.070159 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.070125    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"321675d9-08c8-478d-94e6-e574a875fd3b","Type":"ContainerDied","Data":"6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613"}
Oct 30 11:27:20.070272 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.070198    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"321675d9-08c8-478d-94e6-e574a875fd3b","Type":"ContainerDied","Data":"79aca9a5e3ccf224fb6c7c9c348950be9520fbf8ae847a03d8865c87314adec0"}
Oct 30 11:27:20.070272 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.070220    4849 scope.go:122] "RemoveContainer" containerID="6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613"
Oct 30 11:27:20.071732 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.071716    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:27:20.072812 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.072796    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:27:20.080306 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.080292    4849 scope.go:122] "RemoveContainer" containerID="6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613"
Oct 30 11:27:20.080546 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:27:20.080527    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613\": container with ID starting with 6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613 not found: ID does not exist" containerID="6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613"
Oct 30 11:27:20.080603 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.080551    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613"} err="failed to get container status \"6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613\": rpc error: code = NotFound desc = could not find container \"6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613\": container with ID starting with 6ec23a3bbaeaa2a0ea9435894166c8ec8b129fd892758595e4087ba523d9d613 not found: ID does not exist"
Oct 30 11:27:20.125632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.125608    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:27:20.129921 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.129900    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:27:20.159312 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.159291    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-ec75d9b0b91142f6b6afd54340144a51026b4e387d2663fc875e3386d5d871bf\""
Oct 30 11:27:20.159312 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.159312    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-cde36451543a15adc4b8ffa9b9955623c11fd95bdb37e3b1a03a871bcc3f2ab0\""
Oct 30 11:27:20.179331 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179310    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:27:20.179654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179641    4849 cpu_manager.go:414] "containerMap: removing container" podUID="78abe738-b00b-4b2f-956c-602a8e28294e" containerName="otc-container"
Oct 30 11:27:20.179696 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179658    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78abe738-b00b-4b2f-956c-602a8e28294e" containerName="otc-container"
Oct 30 11:27:20.179696 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179685    4849 cpu_manager.go:414] "containerMap: removing container" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" containerName="loki-ingester"
Oct 30 11:27:20.179696 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179690    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" containerName="loki-ingester"
Oct 30 11:27:20.179780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179704    4849 cpu_manager.go:414] "containerMap: removing container" podUID="5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" containerName="otc-container"
Oct 30 11:27:20.179780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179708    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" containerName="otc-container"
Oct 30 11:27:20.179780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179763    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a3b7d93-4d8a-4be8-b711-e5c9db5980a5" containerName="otc-container"
Oct 30 11:27:20.179780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179774    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="78abe738-b00b-4b2f-956c-602a8e28294e" containerName="otc-container"
Oct 30 11:27:20.179903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.179783    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" containerName="loki-ingester"
Oct 30 11:27:20.184061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.184046    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.200680 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.200662    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-grpc\""
Oct 30 11:27:20.200784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.200725    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-http\""
Oct 30 11:27:20.206869 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.206850    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:27:20.260619 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.260567    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwcwx\" (UniqueName: \"kubernetes.io/projected/77320e79-cdb8-4f7a-b81a-145845f8a87a-kube-api-access-vwcwx\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.260700 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.260617    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.260700 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.260653    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-test-loki-lifecycle-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.260774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.260714    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.260774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.260740    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.260858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.260846    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362009 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.361987    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.362015    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-test-loki-lifecycle-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.362056    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.362077    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362247 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.362130    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362247 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.362147    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwcwx\" (UniqueName: \"kubernetes.io/projected/77320e79-cdb8-4f7a-b81a-145845f8a87a-kube-api-access-vwcwx\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.362851    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.362873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.362861    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.364454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.364409    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.364553 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.364519    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.364606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.364582    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-test-loki-lifecycle-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.386379 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.386360    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwcwx\" (UniqueName: \"kubernetes.io/projected/77320e79-cdb8-4f7a-b81a-145845f8a87a-kube-api-access-vwcwx\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:20.754010 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:20.753986    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="321675d9-08c8-478d-94e6-e574a875fd3b" path="/var/lib/kubelet/pods/321675d9-08c8-478d-94e6-e574a875fd3b/volumes"
Oct 30 11:27:25.400747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.400707    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.400747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.400755    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.446560 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.446522    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.446560 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.446555    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.501905 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.501879    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.502065 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.501917    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.502065 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.502046    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.502198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.502059    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.517870 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.517843    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") DevicePath \"csi-ec75d9b0b91142f6b6afd54340144a51026b4e387d2663fc875e3386d5d871bf\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.517968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.517871    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") DevicePath \"csi-cde36451543a15adc4b8ffa9b9955623c11fd95bdb37e3b1a03a871bcc3f2ab0\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.621925 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.621897    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/1c19039a2608c8bacc8ebf206c5def5c3154a220ec38c0ab0294eb81856ec5cd/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.626044 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.626008    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/f1ba4cd0a8ebf37ed4c6577964876a5661084ff0c33c7df67c3521fff1e49e3c/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.653718 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.653672    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.655172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.655151    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"data-science-lokistack-ingester-0\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:25.894832 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:25.894813    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:26.211000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:26.210927    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:27:26.213443 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:27:26.213412    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77320e79_cdb8_4f7a_b81a_145845f8a87a.slice/crio-9ef181d0d313512565b4cd2092cc92390e5a35411e334b51daabcc113f9ef1d0 WatchSource:0}: Error finding container 9ef181d0d313512565b4cd2092cc92390e5a35411e334b51daabcc113f9ef1d0: Status 404 returned error can't find the container with id 9ef181d0d313512565b4cd2092cc92390e5a35411e334b51daabcc113f9ef1d0
Oct 30 11:27:27.110945 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:27.110908    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"77320e79-cdb8-4f7a-b81a-145845f8a87a","Type":"ContainerStarted","Data":"1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74"}
Oct 30 11:27:27.111353 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:27.110965    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"77320e79-cdb8-4f7a-b81a-145845f8a87a","Type":"ContainerStarted","Data":"9ef181d0d313512565b4cd2092cc92390e5a35411e334b51daabcc113f9ef1d0"}
Oct 30 11:27:27.111353 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:27.110990    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:27:27.158634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:27.158584    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podStartSLOduration=7.158570008 podStartE2EDuration="7.158570008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:27:27.156452405 +0000 UTC m=+2544.536901845" watchObservedRunningTime="2026-10-30 11:27:27.158570008 +0000 UTC m=+2544.539019448"
Oct 30 11:27:39.431226 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:27:39.431190    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:27:48.121762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:48.121662    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="77320e79-cdb8-4f7a-b81a-145845f8a87a" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:27:58.120473 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:27:58.120426    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="77320e79-cdb8-4f7a-b81a-145845f8a87a" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:28:08.120095 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:08.120061    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:28:09.432591 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:28:09.432554    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:28:16.654705 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.654671    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"]
Oct 30 11:28:16.660208 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.660139    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:16.690789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.690762    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-dockercfg-nvp5k\""
Oct 30 11:28:16.690789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.690777    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-6a6878d3\""
Oct 30 11:28:16.693979 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.693956    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"]
Oct 30 11:28:16.799462 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.799435    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/64a9322c-4bc7-4046-9e10-0219c40fbaee-otc-internal\") pod \"data-science-usage-logs-collector-5896c48b7-tbl7h\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:16.799596 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.799518    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppdph\" (UniqueName: \"kubernetes.io/projected/64a9322c-4bc7-4046-9e10-0219c40fbaee-kube-api-access-ppdph\") pod \"data-science-usage-logs-collector-5896c48b7-tbl7h\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:16.900403 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.900380    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/64a9322c-4bc7-4046-9e10-0219c40fbaee-otc-internal\") pod \"data-science-usage-logs-collector-5896c48b7-tbl7h\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:16.900504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.900434    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppdph\" (UniqueName: \"kubernetes.io/projected/64a9322c-4bc7-4046-9e10-0219c40fbaee-kube-api-access-ppdph\") pod \"data-science-usage-logs-collector-5896c48b7-tbl7h\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:16.900982 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.900962    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/64a9322c-4bc7-4046-9e10-0219c40fbaee-otc-internal\") pod \"data-science-usage-logs-collector-5896c48b7-tbl7h\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:16.924944 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.924883    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppdph\" (UniqueName: \"kubernetes.io/projected/64a9322c-4bc7-4046-9e10-0219c40fbaee-kube-api-access-ppdph\") pod \"data-science-usage-logs-collector-5896c48b7-tbl7h\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:16.974894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:16.974872    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:17.277575 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:17.277549    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"]
Oct 30 11:28:17.279709 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:28:17.279678    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a9322c_4bc7_4046_9e10_0219c40fbaee.slice/crio-da19be984c41f99f565fc9b7087c2234a9a501c44df0acf903b2301e2a206261 WatchSource:0}: Error finding container da19be984c41f99f565fc9b7087c2234a9a501c44df0acf903b2301e2a206261: Status 404 returned error can't find the container with id da19be984c41f99f565fc9b7087c2234a9a501c44df0acf903b2301e2a206261
Oct 30 11:28:17.281894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:17.281861    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:28:17.281969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:17.281922    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:28:17.390196 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:17.390156    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h" event={"ID":"64a9322c-4bc7-4046-9e10-0219c40fbaee","Type":"ContainerStarted","Data":"94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7"}
Oct 30 11:28:17.390348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:17.390209    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h" event={"ID":"64a9322c-4bc7-4046-9e10-0219c40fbaee","Type":"ContainerStarted","Data":"da19be984c41f99f565fc9b7087c2234a9a501c44df0acf903b2301e2a206261"}
Oct 30 11:28:17.435174 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:17.435131    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h" podStartSLOduration=1.435116367 podStartE2EDuration="1.435116367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:28:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:28:17.433300008 +0000 UTC m=+2594.813749447" watchObservedRunningTime="2026-10-30 11:28:17.435116367 +0000 UTC m=+2594.815565876"
Oct 30 11:28:18.971256 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:18.971224    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:28:18.971759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:18.971458    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" podUID="bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" containerName="loki-index-gateway" containerID="cri-o://4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908" gracePeriod=30
Oct 30 11:28:19.045353 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.045324    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:28:19.045569 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.045527    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="77320e79-cdb8-4f7a-b81a-145845f8a87a" containerName="loki-ingester" containerID="cri-o://1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74" gracePeriod=600
Oct 30 11:28:19.075160 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.075126    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"]
Oct 30 11:28:19.075365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.075340    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f" podUID="23f37a36-405c-477a-943a-2126b21f7d23" containerName="loki-query-frontend" containerID="cri-o://957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a" gracePeriod=30
Oct 30 11:28:19.078757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.078506    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"]
Oct 30 11:28:19.079159 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.079128    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw" podUID="39eee566-f2de-4c32-87b0-1ad6b34e4a2c" containerName="loki-querier" containerID="cri-o://78f12af48586580b13cc12b3291586b12453859aeb4d736a94e0c40e601fb191" gracePeriod=30
Oct 30 11:28:19.176746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.176716    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"]
Oct 30 11:28:19.283635 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.283611    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:28:19.351433 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.351411    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:28:19.402808 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.402782    4849 generic.go:356] "Generic (PLEG): container finished" podID="39eee566-f2de-4c32-87b0-1ad6b34e4a2c" containerID="78f12af48586580b13cc12b3291586b12453859aeb4d736a94e0c40e601fb191" exitCode=0
Oct 30 11:28:19.402925 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.402859    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw" event={"ID":"39eee566-f2de-4c32-87b0-1ad6b34e4a2c","Type":"ContainerDied","Data":"78f12af48586580b13cc12b3291586b12453859aeb4d736a94e0c40e601fb191"}
Oct 30 11:28:19.404350 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.404327    4849 generic.go:356] "Generic (PLEG): container finished" podID="23f37a36-405c-477a-943a-2126b21f7d23" containerID="957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a" exitCode=0
Oct 30 11:28:19.404457 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.404371    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"
Oct 30 11:28:19.404457 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.404412    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f" event={"ID":"23f37a36-405c-477a-943a-2126b21f7d23","Type":"ContainerDied","Data":"957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a"}
Oct 30 11:28:19.404571 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.404459    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f" event={"ID":"23f37a36-405c-477a-943a-2126b21f7d23","Type":"ContainerDied","Data":"51f2d4122c79ae8dd036e22e9f1c062a2af88ac7b9070ddd7de2829e6085e2ed"}
Oct 30 11:28:19.404571 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.404488    4849 scope.go:122] "RemoveContainer" containerID="957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a"
Oct 30 11:28:19.406101 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.406081    4849 generic.go:356] "Generic (PLEG): container finished" podID="bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" containerID="4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908" exitCode=0
Oct 30 11:28:19.406217 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.406131    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1"
Oct 30 11:28:19.406217 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.406169    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd","Type":"ContainerDied","Data":"4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908"}
Oct 30 11:28:19.406217 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.406205    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-index-gateway-1" event={"ID":"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd","Type":"ContainerDied","Data":"d39c038486b2334e3a74f80dbc03243b6dc2169055cf6c5d0e8d0327af20daa0"}
Oct 30 11:28:19.406372 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.406351    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h" podUID="64a9322c-4bc7-4046-9e10-0219c40fbaee" containerName="otc-container" containerID="cri-o://94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7" gracePeriod=30
Oct 30 11:28:19.422586 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.422565    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") pod \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") "
Oct 30 11:28:19.422701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.422609    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-test-loki-lifecycle-secret\") pod \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") "
Oct 30 11:28:19.422701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.422657    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-ca-bundle\") pod \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") "
Oct 30 11:28:19.422701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.422697    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-kube-api-access-fmkcz\" (UniqueName: \"kubernetes.io/projected/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-kube-api-access-fmkcz\") pod \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") "
Oct 30 11:28:19.422860 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.422755    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-http\") pod \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") "
Oct 30 11:28:19.422860 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.422792    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-grpc\") pod \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") "
Oct 30 11:28:19.422967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.422885    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-config\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-config\") pod \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\" (UID: \"bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd\") "
Oct 30 11:28:19.423499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.423411    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-ca-bundle" pod "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" (UID: "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:19.423499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.423425    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-config" pod "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" (UID: "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:19.425226 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.425200    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-test-loki-lifecycle-secret" pod "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" (UID: "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.426897 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.426873    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015" pod "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" (UID: "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd"). InnerVolumeSpecName "pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:28:19.427262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.427240    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-http" pod "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" (UID: "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd"). InnerVolumeSpecName "data-science-lokistack-index-gateway-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.427334 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.427293    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-kube-api-access-fmkcz" pod "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" (UID: "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd"). InnerVolumeSpecName "kube-api-access-fmkcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:28:19.427607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.427583    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-grpc" pod "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" (UID: "bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd"). InnerVolumeSpecName "data-science-lokistack-index-gateway-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.474833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.474804    4849 scope.go:122] "RemoveContainer" containerID="957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a"
Oct 30 11:28:19.475237 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:28:19.475208    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a\": container with ID starting with 957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a not found: ID does not exist" containerID="957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a"
Oct 30 11:28:19.475318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.475250    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a"} err="failed to get container status \"957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a\": rpc error: code = NotFound desc = could not find container \"957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a\": container with ID starting with 957cee112a0e0e74a53fcbd62e03526efdc5e2e5b0ddb97629e094497f9d959a not found: ID does not exist"
Oct 30 11:28:19.475318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.475274    4849 scope.go:122] "RemoveContainer" containerID="4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908"
Oct 30 11:28:19.517643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.517618    4849 scope.go:122] "RemoveContainer" containerID="4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908"
Oct 30 11:28:19.517977 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:28:19.517949    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908\": container with ID starting with 4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908 not found: ID does not exist" containerID="4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908"
Oct 30 11:28:19.518127 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.517985    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908"} err="failed to get container status \"4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908\": rpc error: code = NotFound desc = could not find container \"4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908\": container with ID starting with 4f21feb651b4369a9035158c5bd4001470295670f0649ad8e9046b185e66d908 not found: ID does not exist"
Oct 30 11:28:19.523372 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523353    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-config\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-config\") pod \"23f37a36-405c-477a-943a-2126b21f7d23\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") "
Oct 30 11:28:19.523492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523425    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-grpc\") pod \"23f37a36-405c-477a-943a-2126b21f7d23\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") "
Oct 30 11:28:19.523492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523468    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-ca-bundle\") pod \"23f37a36-405c-477a-943a-2126b21f7d23\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") "
Oct 30 11:28:19.523606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523496    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23f37a36-405c-477a-943a-2126b21f7d23-kube-api-access-59svq\" (UniqueName: \"kubernetes.io/projected/23f37a36-405c-477a-943a-2126b21f7d23-kube-api-access-59svq\") pod \"23f37a36-405c-477a-943a-2126b21f7d23\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") "
Oct 30 11:28:19.523606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523533    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-http\") pod \"23f37a36-405c-477a-943a-2126b21f7d23\" (UID: \"23f37a36-405c-477a-943a-2126b21f7d23\") "
Oct 30 11:28:19.523809 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523782    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-config" pod "23f37a36-405c-477a-943a-2126b21f7d23" (UID: "23f37a36-405c-477a-943a-2126b21f7d23"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:19.523879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523798    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.523879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523821    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-ca-bundle" pod "23f37a36-405c-477a-943a-2126b21f7d23" (UID: "23f37a36-405c-477a-943a-2126b21f7d23"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:19.523879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523845    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:28:19.523879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523862    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-test-loki-lifecycle-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.523879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523876    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.524134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523886    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmkcz\" (UniqueName: \"kubernetes.io/projected/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-kube-api-access-fmkcz\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.524134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523895    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.524134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.523905    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd-data-science-lokistack-index-gateway-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.525772 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.525751    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-grpc" pod "23f37a36-405c-477a-943a-2126b21f7d23" (UID: "23f37a36-405c-477a-943a-2126b21f7d23"). InnerVolumeSpecName "data-science-lokistack-query-frontend-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.526005 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.525980    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:28:19.526005 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.525990    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-http" pod "23f37a36-405c-477a-943a-2126b21f7d23" (UID: "23f37a36-405c-477a-943a-2126b21f7d23"). InnerVolumeSpecName "data-science-lokistack-query-frontend-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.526156 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.525993    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23f37a36-405c-477a-943a-2126b21f7d23-kube-api-access-59svq" pod "23f37a36-405c-477a-943a-2126b21f7d23" (UID: "23f37a36-405c-477a-943a-2126b21f7d23"). InnerVolumeSpecName "kube-api-access-59svq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:28:19.533628 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.533613    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:28:19.624425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624397    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-http\") pod \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") "
Oct 30 11:28:19.624563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624451    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-kube-api-access-88lz5\" (UniqueName: \"kubernetes.io/projected/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-kube-api-access-88lz5\") pod \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") "
Oct 30 11:28:19.624563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624501    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-config\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-config\") pod \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") "
Oct 30 11:28:19.624563 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624540    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-ca-bundle\") pod \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") "
Oct 30 11:28:19.624743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624594    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-grpc\") pod \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") "
Oct 30 11:28:19.624743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624627    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-test-loki-lifecycle-secret\") pod \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\" (UID: \"39eee566-f2de-4c32-87b0-1ad6b34e4a2c\") "
Oct 30 11:28:19.624901 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624880    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.624967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624906    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59svq\" (UniqueName: \"kubernetes.io/projected/23f37a36-405c-477a-943a-2126b21f7d23-kube-api-access-59svq\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.624967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624911    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-config" pod "39eee566-f2de-4c32-87b0-1ad6b34e4a2c" (UID: "39eee566-f2de-4c32-87b0-1ad6b34e4a2c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:19.624967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624924    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.624967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624949    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-ca-bundle" pod "39eee566-f2de-4c32-87b0-1ad6b34e4a2c" (UID: "39eee566-f2de-4c32-87b0-1ad6b34e4a2c"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:19.625224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624966    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-8dfd52e7-ed74-4fe2-b5b1-118c5d7c74a9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c1499c5fb7eaf015\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-599188236b712b00d635aa56bbdfd3e8788ebbbf3062cdbf2fb2dbf91bd4ee6b\""
Oct 30 11:28:19.625224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.624984    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23f37a36-405c-477a-943a-2126b21f7d23-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.625224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.625002    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/23f37a36-405c-477a-943a-2126b21f7d23-data-science-lokistack-query-frontend-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.627738 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.627699    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-test-loki-lifecycle-secret" pod "39eee566-f2de-4c32-87b0-1ad6b34e4a2c" (UID: "39eee566-f2de-4c32-87b0-1ad6b34e4a2c"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.627841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.627773    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-kube-api-access-88lz5" pod "39eee566-f2de-4c32-87b0-1ad6b34e4a2c" (UID: "39eee566-f2de-4c32-87b0-1ad6b34e4a2c"). InnerVolumeSpecName "kube-api-access-88lz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:28:19.627841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.627783    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-http" pod "39eee566-f2de-4c32-87b0-1ad6b34e4a2c" (UID: "39eee566-f2de-4c32-87b0-1ad6b34e4a2c"). InnerVolumeSpecName "data-science-lokistack-querier-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.627841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.627808    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-grpc" pod "39eee566-f2de-4c32-87b0-1ad6b34e4a2c" (UID: "39eee566-f2de-4c32-87b0-1ad6b34e4a2c"). InnerVolumeSpecName "data-science-lokistack-querier-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:19.658351 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.658330    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:19.725888 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.725861    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.725983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.725895    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.725983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.725913    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.725983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.725929    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-test-loki-lifecycle-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.725983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.725944    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-data-science-lokistack-querier-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.725983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.725958    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88lz5\" (UniqueName: \"kubernetes.io/projected/39eee566-f2de-4c32-87b0-1ad6b34e4a2c-kube-api-access-88lz5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.755736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.755711    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"]
Oct 30 11:28:19.759975 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.759956    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-fl67f"]
Oct 30 11:28:19.797300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.797256    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:28:19.801706 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.801686    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-index-gateway-1"]
Oct 30 11:28:19.827222 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.827200    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64a9322c-4bc7-4046-9e10-0219c40fbaee-otc-internal\" (UniqueName: \"kubernetes.io/configmap/64a9322c-4bc7-4046-9e10-0219c40fbaee-otc-internal\") pod \"64a9322c-4bc7-4046-9e10-0219c40fbaee\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") "
Oct 30 11:28:19.827292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.827269    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64a9322c-4bc7-4046-9e10-0219c40fbaee-kube-api-access-ppdph\" (UniqueName: \"kubernetes.io/projected/64a9322c-4bc7-4046-9e10-0219c40fbaee-kube-api-access-ppdph\") pod \"64a9322c-4bc7-4046-9e10-0219c40fbaee\" (UID: \"64a9322c-4bc7-4046-9e10-0219c40fbaee\") "
Oct 30 11:28:19.827571 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.827552    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64a9322c-4bc7-4046-9e10-0219c40fbaee-otc-internal" pod "64a9322c-4bc7-4046-9e10-0219c40fbaee" (UID: "64a9322c-4bc7-4046-9e10-0219c40fbaee"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:19.829186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.829166    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a9322c-4bc7-4046-9e10-0219c40fbaee-kube-api-access-ppdph" pod "64a9322c-4bc7-4046-9e10-0219c40fbaee" (UID: "64a9322c-4bc7-4046-9e10-0219c40fbaee"). InnerVolumeSpecName "kube-api-access-ppdph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:28:19.927961 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.927939    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/64a9322c-4bc7-4046-9e10-0219c40fbaee-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:19.927961 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:19.927961    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppdph\" (UniqueName: \"kubernetes.io/projected/64a9322c-4bc7-4046-9e10-0219c40fbaee-kube-api-access-ppdph\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:20.191547 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.191523    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:28:20.331126 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331062    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-config\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-config\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331126 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331100    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77320e79-cdb8-4f7a-b81a-145845f8a87a-kube-api-access-vwcwx\" (UniqueName: \"kubernetes.io/projected/77320e79-cdb8-4f7a-b81a-145845f8a87a-kube-api-access-vwcwx\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331187    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331250    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331280    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-grpc\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331323    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-http\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331551 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331372    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ca-bundle\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331551 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331415    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-config" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:20.331551 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331432    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-test-loki-lifecycle-secret\") pod \"77320e79-cdb8-4f7a-b81a-145845f8a87a\" (UID: \"77320e79-cdb8-4f7a-b81a-145845f8a87a\") "
Oct 30 11:28:20.331809 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331694    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:20.331868 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.331821    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ca-bundle" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:28:20.333599 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.333554    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77320e79-cdb8-4f7a-b81a-145845f8a87a-kube-api-access-vwcwx" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "kube-api-access-vwcwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:28:20.333752 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.333729    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-grpc" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "data-science-lokistack-ingester-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:20.333998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.333967    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-http" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "data-science-lokistack-ingester-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:20.334105 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.334079    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-test-loki-lifecycle-secret" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:28:20.334548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.334527    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:28:20.334548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.334537    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f" pod "77320e79-cdb8-4f7a-b81a-145845f8a87a" (UID: "77320e79-cdb8-4f7a-b81a-145845f8a87a"). InnerVolumeSpecName "pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:28:20.412354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.412327    4849 generic.go:356] "Generic (PLEG): container finished" podID="64a9322c-4bc7-4046-9e10-0219c40fbaee" containerID="94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7" exitCode=0
Oct 30 11:28:20.412481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.412368    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"
Oct 30 11:28:20.412481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.412415    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h" event={"ID":"64a9322c-4bc7-4046-9e10-0219c40fbaee","Type":"ContainerDied","Data":"94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7"}
Oct 30 11:28:20.412481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.412464    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h" event={"ID":"64a9322c-4bc7-4046-9e10-0219c40fbaee","Type":"ContainerDied","Data":"da19be984c41f99f565fc9b7087c2234a9a501c44df0acf903b2301e2a206261"}
Oct 30 11:28:20.412629 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.412486    4849 scope.go:122] "RemoveContainer" containerID="94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7"
Oct 30 11:28:20.413918 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.413902    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"
Oct 30 11:28:20.414005 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.413931    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw" event={"ID":"39eee566-f2de-4c32-87b0-1ad6b34e4a2c","Type":"ContainerDied","Data":"71f66e1de7dfe05ad53d93b26ee30ed7c7ff1d79a2b7bc3a771466c718d34d1b"}
Oct 30 11:28:20.417427 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.417406    4849 generic.go:356] "Generic (PLEG): container finished" podID="77320e79-cdb8-4f7a-b81a-145845f8a87a" containerID="1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74" exitCode=0
Oct 30 11:28:20.417523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.417437    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"77320e79-cdb8-4f7a-b81a-145845f8a87a","Type":"ContainerDied","Data":"1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74"}
Oct 30 11:28:20.417523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.417474    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"77320e79-cdb8-4f7a-b81a-145845f8a87a","Type":"ContainerDied","Data":"9ef181d0d313512565b4cd2092cc92390e5a35411e334b51daabcc113f9ef1d0"}
Oct 30 11:28:20.417523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.417519    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 11:28:20.421556 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.421539    4849 scope.go:122] "RemoveContainer" containerID="94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7"
Oct 30 11:28:20.421812 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:28:20.421793    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7\": container with ID starting with 94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7 not found: ID does not exist" containerID="94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7"
Oct 30 11:28:20.421892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.421821    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7"} err="failed to get container status \"94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7\": rpc error: code = NotFound desc = could not find container \"94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7\": container with ID starting with 94c76a16d41c853edcc8f980abcacbd067b2d4779e48f1d42b0708cf145cf6b7 not found: ID does not exist"
Oct 30 11:28:20.421892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.421841    4849 scope.go:122] "RemoveContainer" containerID="78f12af48586580b13cc12b3291586b12453859aeb4d736a94e0c40e601fb191"
Oct 30 11:28:20.430817 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.430801    4849 scope.go:122] "RemoveContainer" containerID="1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74"
Oct 30 11:28:20.432872 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.432856    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-test-loki-lifecycle-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:20.432935 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.432874    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwcwx\" (UniqueName: \"kubernetes.io/projected/77320e79-cdb8-4f7a-b81a-145845f8a87a-kube-api-access-vwcwx\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:20.432935 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.432900    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:28:20.432935 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.432913    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:28:20.432935 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.432923    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:20.433165 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.432934    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ingester-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:20.433165 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.432948    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77320e79-cdb8-4f7a-b81a-145845f8a87a-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:28:20.439385 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.439371    4849 scope.go:122] "RemoveContainer" containerID="1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74"
Oct 30 11:28:20.439655 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:28:20.439638    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74\": container with ID starting with 1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74 not found: ID does not exist" containerID="1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74"
Oct 30 11:28:20.439695 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.439664    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74"} err="failed to get container status \"1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74\": rpc error: code = NotFound desc = could not find container \"1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74\": container with ID starting with 1ebfa971c7d7c781bbffcb08abcc11c5e624d24b8217a41fd4d04b686b292c74 not found: ID does not exist"
Oct 30 11:28:20.452877 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.452863    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:28:20.468087 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.468058    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"]
Oct 30 11:28:20.472014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.471993    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-7dclw"]
Oct 30 11:28:20.476395 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.476381    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:28:20.508479 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.508456    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:28:20.514706 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.514687    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 11:28:20.534221 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.534196    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-7500c3e4-8eda-4feb-ad73-ef4f65175670\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0c7411922d57e0ce3\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-cde36451543a15adc4b8ffa9b9955623c11fd95bdb37e3b1a03a871bcc3f2ab0\""
Oct 30 11:28:20.534303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.534220    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-8b8fadbe-9e8f-4c46-82cd-d227d32b0511\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02dbd7be16ed1d85f\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-ec75d9b0b91142f6b6afd54340144a51026b4e387d2663fc875e3386d5d871bf\""
Oct 30 11:28:20.550765 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.550742    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"]
Oct 30 11:28:20.555132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.555114    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-5896c48b7-tbl7h"]
Oct 30 11:28:20.752909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.752840    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23f37a36-405c-477a-943a-2126b21f7d23" path="/var/lib/kubelet/pods/23f37a36-405c-477a-943a-2126b21f7d23/volumes"
Oct 30 11:28:20.753317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.753297    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39eee566-f2de-4c32-87b0-1ad6b34e4a2c" path="/var/lib/kubelet/pods/39eee566-f2de-4c32-87b0-1ad6b34e4a2c/volumes"
Oct 30 11:28:20.753716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.753696    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64a9322c-4bc7-4046-9e10-0219c40fbaee" path="/var/lib/kubelet/pods/64a9322c-4bc7-4046-9e10-0219c40fbaee/volumes"
Oct 30 11:28:20.754436 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.754414    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77320e79-cdb8-4f7a-b81a-145845f8a87a" path="/var/lib/kubelet/pods/77320e79-cdb8-4f7a-b81a-145845f8a87a/volumes"
Oct 30 11:28:20.754867 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:20.754847    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" path="/var/lib/kubelet/pods/bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd/volumes"
Oct 30 11:28:23.615789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.615758    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"]
Oct 30 11:28:23.616285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616255    4849 cpu_manager.go:414] "containerMap: removing container" podUID="bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" containerName="loki-index-gateway"
Oct 30 11:28:23.616285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616270    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" containerName="loki-index-gateway"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616298    4849 cpu_manager.go:414] "containerMap: removing container" podUID="39eee566-f2de-4c32-87b0-1ad6b34e4a2c" containerName="loki-querier"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616307    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39eee566-f2de-4c32-87b0-1ad6b34e4a2c" containerName="loki-querier"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616317    4849 cpu_manager.go:414] "containerMap: removing container" podUID="64a9322c-4bc7-4046-9e10-0219c40fbaee" containerName="otc-container"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616324    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64a9322c-4bc7-4046-9e10-0219c40fbaee" containerName="otc-container"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616344    4849 cpu_manager.go:414] "containerMap: removing container" podUID="23f37a36-405c-477a-943a-2126b21f7d23" containerName="loki-query-frontend"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616352    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23f37a36-405c-477a-943a-2126b21f7d23" containerName="loki-query-frontend"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616368    4849 cpu_manager.go:414] "containerMap: removing container" podUID="77320e79-cdb8-4f7a-b81a-145845f8a87a" containerName="loki-ingester"
Oct 30 11:28:23.616419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616375    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77320e79-cdb8-4f7a-b81a-145845f8a87a" containerName="loki-ingester"
Oct 30 11:28:23.616757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616462    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="39eee566-f2de-4c32-87b0-1ad6b34e4a2c" containerName="loki-querier"
Oct 30 11:28:23.616757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616474    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="64a9322c-4bc7-4046-9e10-0219c40fbaee" containerName="otc-container"
Oct 30 11:28:23.616757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616486    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="23f37a36-405c-477a-943a-2126b21f7d23" containerName="loki-query-frontend"
Oct 30 11:28:23.616757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616496    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="77320e79-cdb8-4f7a-b81a-145845f8a87a" containerName="loki-ingester"
Oct 30 11:28:23.616757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.616506    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="bae495a4-3ba8-4348-bb5d-1c5b9adb8bcd" containerName="loki-index-gateway"
Oct 30 11:28:23.621287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.621271    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.640203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.640179    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"]
Oct 30 11:28:23.705601 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.705574    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-distributor-grpc\""
Oct 30 11:28:23.705734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.705614    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-distributor-http\""
Oct 30 11:28:23.705734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.705675    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ca-bundle\""
Oct 30 11:28:23.705838 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.705786    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:28:23.705838 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.705789    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-dockercfg-csr95\""
Oct 30 11:28:23.705914 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.705839    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:28:23.705952 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.705941    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-config\""
Oct 30 11:28:23.756521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.756493    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-grpc\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.756646 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.756531    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.756646 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.756557    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dr77\" (UniqueName: \"kubernetes.io/projected/f4084225-bb7a-4635-a542-f0a0a5874d80-kube-api-access-2dr77\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.756771 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.756699    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-config\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.756829 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.756786    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-http\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.771930 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.771902    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"]
Oct 30 11:28:23.775376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.775358    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:23.797178 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.797158    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"]
Oct 30 11:28:23.808451 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.808430    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-querier-grpc\""
Oct 30 11:28:23.808451 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.808445    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"test-loki-lifecycle-secret\""
Oct 30 11:28:23.808589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.808469    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-querier-http\""
Oct 30 11:28:23.858053 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.858013    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-config\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.858163 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.858118    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-http\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.858163 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.858157    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-grpc\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.858292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.858178    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.858391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.858363    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dr77\" (UniqueName: \"kubernetes.io/projected/f4084225-bb7a-4635-a542-f0a0a5874d80-kube-api-access-2dr77\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.858813 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.858779    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.858916 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.858843    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-config\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.860577 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.860558    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-http\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.861119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.861098    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-grpc\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.886084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.886016    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dr77\" (UniqueName: \"kubernetes.io/projected/f4084225-bb7a-4635-a542-f0a0a5874d80-kube-api-access-2dr77\") pod \"data-science-lokistack-distributor-58947679b7-b9frm\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.915931 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.915904    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"]
Oct 30 11:28:23.920184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.920165    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:23.932529 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.932513    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:23.941100 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.940860    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"]
Oct 30 11:28:23.959302 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.959278    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:23.959415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.959318    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:23.959464 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.959426    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:23.959507 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.959492    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-test-loki-lifecycle-secret\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:23.959544 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.959520    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49vqx\" (UniqueName: \"kubernetes.io/projected/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-kube-api-access-49vqx\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:23.959613 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:23.959601    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-config\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.006389 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.006294    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-query-frontend-grpc\""
Oct 30 11:28:24.006389 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.006368    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-query-frontend-http\""
Oct 30 11:28:24.060711 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.060680    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.060842 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.060718    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.060922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.060845    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.060922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.060877    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-grpc\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.060922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.060899    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-test-loki-lifecycle-secret\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.060922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.060918    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-config\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.061151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.060979    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49vqx\" (UniqueName: \"kubernetes.io/projected/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-kube-api-access-49vqx\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.061151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.061106    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cxcf\" (UniqueName: \"kubernetes.io/projected/441aa7d1-96af-4bcf-a9d9-9aca116380d2-kube-api-access-2cxcf\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.061270 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.061154    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-config\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.061270 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.061184    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-http\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.061427 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.061404    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.061684 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.061651    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.061875 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.061855    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-config\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.063310 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.063288    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.063398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.063376    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.063466 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.063448    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-test-loki-lifecycle-secret\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.090057 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.090010    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49vqx\" (UniqueName: \"kubernetes.io/projected/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-kube-api-access-49vqx\") pod \"data-science-lokistack-querier-56ffbd5fdc-2pcdk\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.162875 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.162815    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-grpc\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.162875 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.162853    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-config\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.163137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.162890    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cxcf\" (UniqueName: \"kubernetes.io/projected/441aa7d1-96af-4bcf-a9d9-9aca116380d2-kube-api-access-2cxcf\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.163137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.162924    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-http\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.163137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.162980    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.163676 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.163648    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.163899 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.163827    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-config\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.165266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.165243    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-http\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.165458 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.165433    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-grpc\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.190608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.190587    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cxcf\" (UniqueName: \"kubernetes.io/projected/441aa7d1-96af-4bcf-a9d9-9aca116380d2-kube-api-access-2cxcf\") pod \"data-science-lokistack-query-frontend-f746fc66d-8tcrs\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") " pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.232290 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.232269    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:24.244955 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.244931    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"]
Oct 30 11:28:24.247013 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:28:24.246991    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4084225_bb7a_4635_a542_f0a0a5874d80.slice/crio-34083befef6aeff3fffbd00921923f56bd2a91d4d32d8ece39858f055a25ee3f WatchSource:0}: Error finding container 34083befef6aeff3fffbd00921923f56bd2a91d4d32d8ece39858f055a25ee3f: Status 404 returned error can't find the container with id 34083befef6aeff3fffbd00921923f56bd2a91d4d32d8ece39858f055a25ee3f
Oct 30 11:28:24.386726 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.386696    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:24.447643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.446627    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm" event={"ID":"f4084225-bb7a-4635-a542-f0a0a5874d80","Type":"ContainerStarted","Data":"01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554"}
Oct 30 11:28:24.447643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.446689    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm" event={"ID":"f4084225-bb7a-4635-a542-f0a0a5874d80","Type":"ContainerStarted","Data":"34083befef6aeff3fffbd00921923f56bd2a91d4d32d8ece39858f055a25ee3f"}
Oct 30 11:28:24.447643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.447582    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:24.493578 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.493525    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm" podStartSLOduration=1.49350991 podStartE2EDuration="1.49350991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:28:24.491442849 +0000 UTC m=+2601.871892288" watchObservedRunningTime="2026-10-30 11:28:24.49350991 +0000 UTC m=+2601.873959348"
Oct 30 11:28:24.543747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.543726    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"]
Oct 30 11:28:24.545674 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:28:24.545648    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441aa7d1_96af_4bcf_a9d9_9aca116380d2.slice/crio-8a28f0b5024ebd9b3d2f2b0095a4411a3a4a3a206be0599420312c06362f283e WatchSource:0}: Error finding container 8a28f0b5024ebd9b3d2f2b0095a4411a3a4a3a206be0599420312c06362f283e: Status 404 returned error can't find the container with id 8a28f0b5024ebd9b3d2f2b0095a4411a3a4a3a206be0599420312c06362f283e
Oct 30 11:28:24.705142 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:24.705113    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"]
Oct 30 11:28:24.707461 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:28:24.707431    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ff1bc1_63c8_4753_833f_3f3c43e60a51.slice/crio-86ba939c6ceb704e9379b82b2276e2da9dec601de64064fa3abf0981160d9b96 WatchSource:0}: Error finding container 86ba939c6ceb704e9379b82b2276e2da9dec601de64064fa3abf0981160d9b96: Status 404 returned error can't find the container with id 86ba939c6ceb704e9379b82b2276e2da9dec601de64064fa3abf0981160d9b96
Oct 30 11:28:25.453301 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:25.453262    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk" event={"ID":"b8ff1bc1-63c8-4753-833f-3f3c43e60a51","Type":"ContainerStarted","Data":"13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd"}
Oct 30 11:28:25.453492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:25.453323    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk" event={"ID":"b8ff1bc1-63c8-4753-833f-3f3c43e60a51","Type":"ContainerStarted","Data":"86ba939c6ceb704e9379b82b2276e2da9dec601de64064fa3abf0981160d9b96"}
Oct 30 11:28:25.453492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:25.453347    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:25.454892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:25.454853    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs" event={"ID":"441aa7d1-96af-4bcf-a9d9-9aca116380d2","Type":"ContainerStarted","Data":"26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e"}
Oct 30 11:28:25.455009 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:25.454908    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs" event={"ID":"441aa7d1-96af-4bcf-a9d9-9aca116380d2","Type":"ContainerStarted","Data":"8a28f0b5024ebd9b3d2f2b0095a4411a3a4a3a206be0599420312c06362f283e"}
Oct 30 11:28:25.499938 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:25.499894    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk" podStartSLOduration=2.49988209 podStartE2EDuration="2.49988209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:28:25.497361564 +0000 UTC m=+2602.877811004" watchObservedRunningTime="2026-10-30 11:28:25.49988209 +0000 UTC m=+2602.880331528"
Oct 30 11:28:26.460752 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:26.460713    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:28:39.430740 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:28:39.430703    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:28:46.465525 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:46.465493    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:28:46.466066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:46.466047    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:28:46.512591 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:46.512542    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs" podStartSLOduration=23.512529448 podStartE2EDuration="23.512529448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:28:25.590679424 +0000 UTC m=+2602.971128863" watchObservedRunningTime="2026-10-30 11:28:46.512529448 +0000 UTC m=+2623.892978887"
Oct 30 11:28:47.471367 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:28:47.471335    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:29:09.432189 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:29:09.432138    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:29:39.429790 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:29:39.429712    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:30:03.053168 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:03.053060    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:30:03.063251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:03.063227    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:30:09.327185 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.327145    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-2"]
Oct 30 11:30:09.330917 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.330895    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.360837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.360814    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-grpc\""
Oct 30 11:30:09.360837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.360817    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-http\""
Oct 30 11:30:09.362788 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.362768    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-2"]
Oct 30 11:30:09.429596 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:30:09.429566    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:30:09.452977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.452954    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.453119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.452985    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-test-loki-lifecycle-secret\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.453119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.453014    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.453199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.453158    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.453199 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.453195    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-config\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.453267 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.453229    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlr6p\" (UniqueName: \"kubernetes.io/projected/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-kube-api-access-zlr6p\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.553798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.553770    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.553894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.553828    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-config\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.553894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.553862    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlr6p\" (UniqueName: \"kubernetes.io/projected/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-kube-api-access-zlr6p\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.553967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.553921    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.553967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.553951    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-test-loki-lifecycle-secret\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.554093 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.553997    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.554700 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.554678    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.554817 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.554726    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-config\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.556051 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.556018    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.556320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.556305    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.556745 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.556726    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-test-loki-lifecycle-secret\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:09.581261 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:09.581208    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlr6p\" (UniqueName: \"kubernetes.io/projected/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-kube-api-access-zlr6p\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.431467 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.431434    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.431467 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.431473    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.464096 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.464069    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.464260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.464163    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.532520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.532495    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.532612 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.532531    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.532663 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.532628    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.532713 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.532631    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.548564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.548539    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") DevicePath \"csi-09c98fdea603292ff891abff2e429549f1cfdecfe466a76adfa0219eaa95f889\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.548564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.548558    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") DevicePath \"csi-cf9063396e29567420f2a385a73136d285f62279ef3637e67b81e2016fc24557\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.654340 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.654310    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b554cdd3f05b8fbf122d7cc95d00498b97c61b4880dd57285f4e89e0d822aafe/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.661282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.661260    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/77ed136e3dd92e8583c89d43615042d6e129a92ae818289ba507b9d44ff2235f/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.686594 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.686547    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.688178 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.688155    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"data-science-lokistack-ingester-2\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:19.841846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:19.841826    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:20.159053 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:20.159012    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-2"]
Oct 30 11:30:20.161285 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:30:20.161257    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7f08b4_e11d_4c25_bd1f_b9a89dae5583.slice/crio-762ec6305dfe723ef89226c62086047e357833f0af1ea9dcc69f7fe3e51cf8c8 WatchSource:0}: Error finding container 762ec6305dfe723ef89226c62086047e357833f0af1ea9dcc69f7fe3e51cf8c8: Status 404 returned error can't find the container with id 762ec6305dfe723ef89226c62086047e357833f0af1ea9dcc69f7fe3e51cf8c8
Oct 30 11:30:21.043954 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:21.043916    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" event={"ID":"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583","Type":"ContainerStarted","Data":"4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75"}
Oct 30 11:30:21.044391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:21.043976    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" event={"ID":"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583","Type":"ContainerStarted","Data":"762ec6305dfe723ef89226c62086047e357833f0af1ea9dcc69f7fe3e51cf8c8"}
Oct 30 11:30:21.044391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:21.044130    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:30:21.094278 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:21.094234    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" podStartSLOduration=12.094222568 podStartE2EDuration="12.094222568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:30:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:30:21.091763801 +0000 UTC m=+2718.472213240" watchObservedRunningTime="2026-10-30 11:30:21.094222568 +0000 UTC m=+2718.474671998"
Oct 30 11:30:39.430748 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:30:39.430714    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:30:42.054937 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:42.054894    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" podUID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:30:52.054010 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:30:52.053964    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" podUID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:31:02.053753 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:02.053725    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:31:09.433950 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:31:09.433912    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:31:34.791951 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.791920    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-2"]
Oct 30 11:31:34.792540 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.792156    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" podUID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" containerName="loki-ingester" containerID="cri-o://4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75" gracePeriod=600
Oct 30 11:31:34.875166 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.875134    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"]
Oct 30 11:31:34.875350 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.875328    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm" podUID="f4084225-bb7a-4635-a542-f0a0a5874d80" containerName="loki-distributor" containerID="cri-o://01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554" gracePeriod=30
Oct 30 11:31:34.880680 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.880653    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"]
Oct 30 11:31:34.880839 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.880819    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk" podUID="b8ff1bc1-63c8-4753-833f-3f3c43e60a51" containerName="loki-querier" containerID="cri-o://13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd" gracePeriod=30
Oct 30 11:31:34.890538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.890102    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"]
Oct 30 11:31:34.890538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:34.890375    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs" podUID="441aa7d1-96af-4bcf-a9d9-9aca116380d2" containerName="loki-query-frontend" containerID="cri-o://26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e" gracePeriod=30
Oct 30 11:31:35.216159 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.216134    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:31:35.220062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.220012    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:31:35.320858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.320803    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/441aa7d1-96af-4bcf-a9d9-9aca116380d2-kube-api-access-2cxcf\" (UniqueName: \"kubernetes.io/projected/441aa7d1-96af-4bcf-a9d9-9aca116380d2-kube-api-access-2cxcf\") pod \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") "
Oct 30 11:31:35.320858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.320834    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-grpc\") pod \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") "
Oct 30 11:31:35.321069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.320861    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-config\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-config\") pod \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") "
Oct 30 11:31:35.321069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.320882    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-grpc\") pod \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") "
Oct 30 11:31:35.321069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.320930    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-kube-api-access-49vqx\" (UniqueName: \"kubernetes.io/projected/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-kube-api-access-49vqx\") pod \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") "
Oct 30 11:31:35.321069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.320948    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-http\") pod \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") "
Oct 30 11:31:35.321069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.320966    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-ca-bundle\") pod \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") "
Oct 30 11:31:35.321069 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.321008    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-config\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-config\") pod \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") "
Oct 30 11:31:35.321404 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.321071    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-ca-bundle\") pod \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\" (UID: \"441aa7d1-96af-4bcf-a9d9-9aca116380d2\") "
Oct 30 11:31:35.321404 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.321124    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-test-loki-lifecycle-secret\") pod \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") "
Oct 30 11:31:35.321404 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.321179    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-http\") pod \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\" (UID: \"b8ff1bc1-63c8-4753-833f-3f3c43e60a51\") "
Oct 30 11:31:35.322377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.321769    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-config" pod "441aa7d1-96af-4bcf-a9d9-9aca116380d2" (UID: "441aa7d1-96af-4bcf-a9d9-9aca116380d2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:35.322377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.321974    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-ca-bundle" pod "441aa7d1-96af-4bcf-a9d9-9aca116380d2" (UID: "441aa7d1-96af-4bcf-a9d9-9aca116380d2"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:35.322377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.322236    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-config" pod "b8ff1bc1-63c8-4753-833f-3f3c43e60a51" (UID: "b8ff1bc1-63c8-4753-833f-3f3c43e60a51"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:35.322377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.322346    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-ca-bundle" pod "b8ff1bc1-63c8-4753-833f-3f3c43e60a51" (UID: "b8ff1bc1-63c8-4753-833f-3f3c43e60a51"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:35.323668 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.323646    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-http" pod "b8ff1bc1-63c8-4753-833f-3f3c43e60a51" (UID: "b8ff1bc1-63c8-4753-833f-3f3c43e60a51"). InnerVolumeSpecName "data-science-lokistack-querier-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.323784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.323669    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-grpc" pod "b8ff1bc1-63c8-4753-833f-3f3c43e60a51" (UID: "b8ff1bc1-63c8-4753-833f-3f3c43e60a51"). InnerVolumeSpecName "data-science-lokistack-querier-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.324317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.324258    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-kube-api-access-49vqx" pod "b8ff1bc1-63c8-4753-833f-3f3c43e60a51" (UID: "b8ff1bc1-63c8-4753-833f-3f3c43e60a51"). InnerVolumeSpecName "kube-api-access-49vqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:31:35.324415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.324364    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-grpc" pod "441aa7d1-96af-4bcf-a9d9-9aca116380d2" (UID: "441aa7d1-96af-4bcf-a9d9-9aca116380d2"). InnerVolumeSpecName "data-science-lokistack-query-frontend-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.324415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.324364    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/441aa7d1-96af-4bcf-a9d9-9aca116380d2-kube-api-access-2cxcf" pod "441aa7d1-96af-4bcf-a9d9-9aca116380d2" (UID: "441aa7d1-96af-4bcf-a9d9-9aca116380d2"). InnerVolumeSpecName "kube-api-access-2cxcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:31:35.325152 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.325124    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-test-loki-lifecycle-secret" pod "b8ff1bc1-63c8-4753-833f-3f3c43e60a51" (UID: "b8ff1bc1-63c8-4753-833f-3f3c43e60a51"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.325243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.325130    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-http" pod "441aa7d1-96af-4bcf-a9d9-9aca116380d2" (UID: "441aa7d1-96af-4bcf-a9d9-9aca116380d2"). InnerVolumeSpecName "data-science-lokistack-query-frontend-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.421320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421290    4849 generic.go:356] "Generic (PLEG): container finished" podID="441aa7d1-96af-4bcf-a9d9-9aca116380d2" containerID="26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e" exitCode=0
Oct 30 11:31:35.421427 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421333    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"
Oct 30 11:31:35.421427 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421363    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs" event={"ID":"441aa7d1-96af-4bcf-a9d9-9aca116380d2","Type":"ContainerDied","Data":"26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e"}
Oct 30 11:31:35.421576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421436    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs" event={"ID":"441aa7d1-96af-4bcf-a9d9-9aca116380d2","Type":"ContainerDied","Data":"8a28f0b5024ebd9b3d2f2b0095a4411a3a4a3a206be0599420312c06362f283e"}
Oct 30 11:31:35.421576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421467    4849 scope.go:122] "RemoveContainer" containerID="26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e"
Oct 30 11:31:35.421932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421910    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421935    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cxcf\" (UniqueName: \"kubernetes.io/projected/441aa7d1-96af-4bcf-a9d9-9aca116380d2-kube-api-access-2cxcf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421952    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-querier-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421966    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421979    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-query-frontend-grpc\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.421993    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49vqx\" (UniqueName: \"kubernetes.io/projected/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-kube-api-access-49vqx\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.422005    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-query-frontend-http\" (UniqueName: \"kubernetes.io/secret/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-query-frontend-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.422014    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.422045    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.422057    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/441aa7d1-96af-4bcf-a9d9-9aca116380d2-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.422080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.422071    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/b8ff1bc1-63c8-4753-833f-3f3c43e60a51-test-loki-lifecycle-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.423000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.422974    4849 generic.go:356] "Generic (PLEG): container finished" podID="b8ff1bc1-63c8-4753-833f-3f3c43e60a51" containerID="13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd" exitCode=0
Oct 30 11:31:35.423125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.423008    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"
Oct 30 11:31:35.423125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.423049    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk" event={"ID":"b8ff1bc1-63c8-4753-833f-3f3c43e60a51","Type":"ContainerDied","Data":"13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd"}
Oct 30 11:31:35.423125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.423097    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk" event={"ID":"b8ff1bc1-63c8-4753-833f-3f3c43e60a51","Type":"ContainerDied","Data":"86ba939c6ceb704e9379b82b2276e2da9dec601de64064fa3abf0981160d9b96"}
Oct 30 11:31:35.431723 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.431689    4849 scope.go:122] "RemoveContainer" containerID="26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e"
Oct 30 11:31:35.432303 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:31:35.431994    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e\": container with ID starting with 26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e not found: ID does not exist" containerID="26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e"
Oct 30 11:31:35.432303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.432049    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e"} err="failed to get container status \"26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e\": rpc error: code = NotFound desc = could not find container \"26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e\": container with ID starting with 26b13e0721308451a870c2fb68a9d840bf12e9993a802c7e0a1455c4db14226e not found: ID does not exist"
Oct 30 11:31:35.432303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.432073    4849 scope.go:122] "RemoveContainer" containerID="13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd"
Oct 30 11:31:35.441382 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.441362    4849 scope.go:122] "RemoveContainer" containerID="13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd"
Oct 30 11:31:35.441646 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:31:35.441622    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd\": container with ID starting with 13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd not found: ID does not exist" containerID="13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd"
Oct 30 11:31:35.441691 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.441656    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd"} err="failed to get container status \"13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd\": rpc error: code = NotFound desc = could not find container \"13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd\": container with ID starting with 13b61844045a04e93b922a828e710cf34b00b2b4718b810d9eff4e815384b6dd not found: ID does not exist"
Oct 30 11:31:35.477693 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.477666    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"]
Oct 30 11:31:35.481913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.481894    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-56ffbd5fdc-2pcdk"]
Oct 30 11:31:35.519936 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.519908    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"]
Oct 30 11:31:35.524432 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.524410    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-query-frontend-f746fc66d-8tcrs"]
Oct 30 11:31:35.746449 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.746426    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:31:35.826599 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826572    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-grpc\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.826929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826688    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.826929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826724    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-test-loki-lifecycle-secret\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.826929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826796    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.826929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826848    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-kube-api-access-zlr6p\" (UniqueName: \"kubernetes.io/projected/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-kube-api-access-zlr6p\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.826929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826881    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-http\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.827356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826933    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ca-bundle\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.827356 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.826965    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-config\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-config\") pod \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\" (UID: \"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583\") "
Oct 30 11:31:35.827521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.827492    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ca-bundle" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:35.827607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.827567    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-config" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:35.828040 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.827971    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.828040 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.827998    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.829416 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.829392    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-grpc" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "data-science-lokistack-ingester-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.829534 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.829446    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-test-loki-lifecycle-secret" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.829577 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.829531    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-http" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "data-science-lokistack-ingester-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:35.829736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.829714    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-kube-api-access-zlr6p" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "kube-api-access-zlr6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:31:35.830099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.830080    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:31:35.830200 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.830186    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed" pod "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" (UID: "ad7f08b4-e11d-4c25-bd1f-b9a89dae5583"). InnerVolumeSpecName "pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:31:35.928990 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.928967    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlr6p\" (UniqueName: \"kubernetes.io/projected/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-kube-api-access-zlr6p\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.928990 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.928990    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.929140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.928999    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-data-science-lokistack-ingester-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.929140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.929045    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:31:35.929140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.929056    4849 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583-test-loki-lifecycle-secret\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:35.929140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.929071    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:31:35.939100 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.939052    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:31:35.942972 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:35.942956    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:31:36.029845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.029818    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-3ce17b64-6920-453b-9cd0-0f4f1cfc878a\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-05e36655afef843ed\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-09c98fdea603292ff891abff2e429549f1cfdecfe466a76adfa0219eaa95f889\""
Oct 30 11:31:36.029845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.029842    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-8e4fa8c4-18d1-47ba-9c31-d641c06074c2\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0d2b13fbb21c9894f\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-cf9063396e29567420f2a385a73136d285f62279ef3637e67b81e2016fc24557\""
Oct 30 11:31:36.430892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.430861    4849 generic.go:356] "Generic (PLEG): container finished" podID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" containerID="4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75" exitCode=0
Oct 30 11:31:36.431056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.430907    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2"
Oct 30 11:31:36.431056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.430942    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" event={"ID":"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583","Type":"ContainerDied","Data":"4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75"}
Oct 30 11:31:36.431056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.430996    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-2" event={"ID":"ad7f08b4-e11d-4c25-bd1f-b9a89dae5583","Type":"ContainerDied","Data":"762ec6305dfe723ef89226c62086047e357833f0af1ea9dcc69f7fe3e51cf8c8"}
Oct 30 11:31:36.431056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.431016    4849 scope.go:122] "RemoveContainer" containerID="4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75"
Oct 30 11:31:36.441550 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.441531    4849 scope.go:122] "RemoveContainer" containerID="4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75"
Oct 30 11:31:36.441833 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:31:36.441813    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75\": container with ID starting with 4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75 not found: ID does not exist" containerID="4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75"
Oct 30 11:31:36.441878 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.441840    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75"} err="failed to get container status \"4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75\": rpc error: code = NotFound desc = could not find container \"4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75\": container with ID starting with 4f3caab3daa2453fbf142590aeaa19aae9820cd009e41fa4cb18dea9367a1e75 not found: ID does not exist"
Oct 30 11:31:36.464576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.464550    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm" podUID="f4084225-bb7a-4635-a542-f0a0a5874d80" containerName="loki-distributor" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 11:31:36.486604 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.486557    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-2"]
Oct 30 11:31:36.499939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.499916    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-2"]
Oct 30 11:31:36.753105 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.753050    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="441aa7d1-96af-4bcf-a9d9-9aca116380d2" path="/var/lib/kubelet/pods/441aa7d1-96af-4bcf-a9d9-9aca116380d2/volumes"
Oct 30 11:31:36.753501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.753486    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" path="/var/lib/kubelet/pods/ad7f08b4-e11d-4c25-bd1f-b9a89dae5583/volumes"
Oct 30 11:31:36.753859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:36.753845    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8ff1bc1-63c8-4753-833f-3f3c43e60a51" path="/var/lib/kubelet/pods/b8ff1bc1-63c8-4753-833f-3f3c43e60a51/volumes"
Oct 30 11:31:37.132486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.132463    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:31:37.238430 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.238402    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-http\") pod \"f4084225-bb7a-4635-a542-f0a0a5874d80\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") "
Oct 30 11:31:37.238548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.238447    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-ca-bundle\") pod \"f4084225-bb7a-4635-a542-f0a0a5874d80\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") "
Oct 30 11:31:37.238548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.238471    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-grpc\") pod \"f4084225-bb7a-4635-a542-f0a0a5874d80\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") "
Oct 30 11:31:37.238548 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.238527    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4084225-bb7a-4635-a542-f0a0a5874d80-kube-api-access-2dr77\" (UniqueName: \"kubernetes.io/projected/f4084225-bb7a-4635-a542-f0a0a5874d80-kube-api-access-2dr77\") pod \"f4084225-bb7a-4635-a542-f0a0a5874d80\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") "
Oct 30 11:31:37.238674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.238572    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-config\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-config\") pod \"f4084225-bb7a-4635-a542-f0a0a5874d80\" (UID: \"f4084225-bb7a-4635-a542-f0a0a5874d80\") "
Oct 30 11:31:37.238786 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.238763    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-ca-bundle" pod "f4084225-bb7a-4635-a542-f0a0a5874d80" (UID: "f4084225-bb7a-4635-a542-f0a0a5874d80"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:37.238856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.238831    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-ca-bundle\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:37.239080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.239057    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-config" pod "f4084225-bb7a-4635-a542-f0a0a5874d80" (UID: "f4084225-bb7a-4635-a542-f0a0a5874d80"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:31:37.240596 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.240569    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-http" pod "f4084225-bb7a-4635-a542-f0a0a5874d80" (UID: "f4084225-bb7a-4635-a542-f0a0a5874d80"). InnerVolumeSpecName "data-science-lokistack-distributor-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:37.240679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.240593    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4084225-bb7a-4635-a542-f0a0a5874d80-kube-api-access-2dr77" pod "f4084225-bb7a-4635-a542-f0a0a5874d80" (UID: "f4084225-bb7a-4635-a542-f0a0a5874d80"). InnerVolumeSpecName "kube-api-access-2dr77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:31:37.240679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.240617    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-grpc" pod "f4084225-bb7a-4635-a542-f0a0a5874d80" (UID: "f4084225-bb7a-4635-a542-f0a0a5874d80"). InnerVolumeSpecName "data-science-lokistack-distributor-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:31:37.340104 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.340081    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dr77\" (UniqueName: \"kubernetes.io/projected/f4084225-bb7a-4635-a542-f0a0a5874d80-kube-api-access-2dr77\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:37.340104 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.340103    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4084225-bb7a-4635-a542-f0a0a5874d80-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:37.340229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.340113    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-http\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:37.340229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.340123    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/f4084225-bb7a-4635-a542-f0a0a5874d80-data-science-lokistack-distributor-grpc\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:31:37.439419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.439394    4849 generic.go:356] "Generic (PLEG): container finished" podID="f4084225-bb7a-4635-a542-f0a0a5874d80" containerID="01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554" exitCode=0
Oct 30 11:31:37.439527 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.439428    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm" event={"ID":"f4084225-bb7a-4635-a542-f0a0a5874d80","Type":"ContainerDied","Data":"01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554"}
Oct 30 11:31:37.439527 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.439441    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"
Oct 30 11:31:37.439527 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.439477    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm" event={"ID":"f4084225-bb7a-4635-a542-f0a0a5874d80","Type":"ContainerDied","Data":"34083befef6aeff3fffbd00921923f56bd2a91d4d32d8ece39858f055a25ee3f"}
Oct 30 11:31:37.439527 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.439507    4849 scope.go:122] "RemoveContainer" containerID="01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554"
Oct 30 11:31:37.448534 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.448510    4849 scope.go:122] "RemoveContainer" containerID="01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554"
Oct 30 11:31:37.448795 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:31:37.448775    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554\": container with ID starting with 01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554 not found: ID does not exist" containerID="01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554"
Oct 30 11:31:37.448856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.448804    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554"} err="failed to get container status \"01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554\": rpc error: code = NotFound desc = could not find container \"01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554\": container with ID starting with 01b737f87d9a46f49328d7fae22e95ce09771d42083f9361af73456f7ef43554 not found: ID does not exist"
Oct 30 11:31:37.498527 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.498503    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"]
Oct 30 11:31:37.512060 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:37.512036    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-58947679b7-b9frm"]
Oct 30 11:31:38.753357 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:31:38.753325    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4084225-bb7a-4635-a542-f0a0a5874d80" path="/var/lib/kubelet/pods/f4084225-bb7a-4635-a542-f0a0a5874d80/volumes"
Oct 30 11:31:39.430691 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:31:39.430653    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:32:09.090547 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090509    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090883    4849 cpu_manager.go:414] "containerMap: removing container" podUID="b8ff1bc1-63c8-4753-833f-3f3c43e60a51" containerName="loki-querier"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090896    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8ff1bc1-63c8-4753-833f-3f3c43e60a51" containerName="loki-querier"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090908    4849 cpu_manager.go:414] "containerMap: removing container" podUID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" containerName="loki-ingester"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090914    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" containerName="loki-ingester"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090928    4849 cpu_manager.go:414] "containerMap: removing container" podUID="f4084225-bb7a-4635-a542-f0a0a5874d80" containerName="loki-distributor"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090933    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4084225-bb7a-4635-a542-f0a0a5874d80" containerName="loki-distributor"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090951    4849 cpu_manager.go:414] "containerMap: removing container" podUID="441aa7d1-96af-4bcf-a9d9-9aca116380d2" containerName="loki-query-frontend"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.090955    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="441aa7d1-96af-4bcf-a9d9-9aca116380d2" containerName="loki-query-frontend"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.091006    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8ff1bc1-63c8-4753-833f-3f3c43e60a51" containerName="loki-querier"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.091014    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad7f08b4-e11d-4c25-bd1f-b9a89dae5583" containerName="loki-ingester"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.091038    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4084225-bb7a-4635-a542-f0a0a5874d80" containerName="loki-distributor"
Oct 30 11:32:09.091092 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.091046    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="441aa7d1-96af-4bcf-a9d9-9aca116380d2" containerName="loki-query-frontend"
Oct 30 11:32:09.095883 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.095862    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.132442 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.132413    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:32:09.132594 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.132500    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:32:09.132594 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.132514    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:32:09.132713 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.132614    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-2cltz\""
Oct 30 11:32:09.132713 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.132671    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:32:09.132791 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.132781    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:32:09.133013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.132998    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:32:09.134328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.134309    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:09.279923 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.279891    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.280094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.279949    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.280094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.280083    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dpr5\" (UniqueName: \"kubernetes.io/projected/df239dd2-c466-4e50-ac02-aef085e28ca2-kube-api-access-9dpr5\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.280180 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.280116    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/df239dd2-c466-4e50-ac02-aef085e28ca2-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.280180 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.280141    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.381543 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.381470    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.381543 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.381507    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dpr5\" (UniqueName: \"kubernetes.io/projected/df239dd2-c466-4e50-ac02-aef085e28ca2-kube-api-access-9dpr5\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.381774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.381536    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/df239dd2-c466-4e50-ac02-aef085e28ca2-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.381774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.381579    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.381774 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.381618    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.382370 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.382343    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/df239dd2-c466-4e50-ac02-aef085e28ca2-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.384136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.384117    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.384233 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.384141    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.385510 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.385488    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.406222 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.406202    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dpr5\" (UniqueName: \"kubernetes.io/projected/df239dd2-c466-4e50-ac02-aef085e28ca2-kube-api-access-9dpr5\") pod \"data-science-collector-collector-0\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.409966 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.409944    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:09.438832 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:32:09.438801    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:32:09.720677 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.720633    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:09.723439 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:32:09.723404    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf239dd2_c466_4e50_ac02_aef085e28ca2.slice/crio-d5590a7b7fe568c0f419b15863db08f48afda2129cc64299f73095f056852d48 WatchSource:0}: Error finding container d5590a7b7fe568c0f419b15863db08f48afda2129cc64299f73095f056852d48: Status 404 returned error can't find the container with id d5590a7b7fe568c0f419b15863db08f48afda2129cc64299f73095f056852d48
Oct 30 11:32:09.725461 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.725425    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:32:09.725538 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:09.725486    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:32:10.619572 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:10.619537    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"df239dd2-c466-4e50-ac02-aef085e28ca2","Type":"ContainerStarted","Data":"2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d"}
Oct 30 11:32:10.619980 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:10.619591    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"df239dd2-c466-4e50-ac02-aef085e28ca2","Type":"ContainerStarted","Data":"d5590a7b7fe568c0f419b15863db08f48afda2129cc64299f73095f056852d48"}
Oct 30 11:32:10.666454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:10.666410    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.666396548 podStartE2EDuration="1.666396548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:32:10.663944519 +0000 UTC m=+2828.044393957" watchObservedRunningTime="2026-10-30 11:32:10.666396548 +0000 UTC m=+2828.046845988"
Oct 30 11:32:12.354542 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.354510    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:32:12.358315 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.358295    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.377138 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.377111    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:32:12.377694 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.377676    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 11:32:12.377788 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.377701    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 11:32:12.389874 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.389853    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 11:32:12.390250 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.390231    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-vp8p7\""
Oct 30 11:32:12.392304 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.392281    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:32:12.417259 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.417232    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.417365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.417265    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.417365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.417297    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.417476 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.417418    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqlhs\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-kube-api-access-lqlhs\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.417524 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.417486    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.417577 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.417528    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.417577 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.417554    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.458666 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.458642    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:32:12.462294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.462280    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.480558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.480535    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 11:32:12.481586 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.481565    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:32:12.481704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.481619    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 11:32:12.481788 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.481573    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 11:32:12.481788 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.481690    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 11:32:12.481950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.481676    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 11:32:12.481950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.481853    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 11:32:12.481950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.481934    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-2lb6p\""
Oct 30 11:32:12.493927 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.493888    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 11:32:12.494094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.493985    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 11:32:12.494279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.494261    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:32:12.518722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.518699    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.518837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.518732    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.518837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.518770    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.518837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.518798    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.518837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.518831    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.519127 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.518858    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqlhs\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-kube-api-access-lqlhs\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.519127 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.518884    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.519282 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.519250    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.521362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.521337    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.521841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.521498    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.521841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.521501    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.521841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.521810    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.522238 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.522218    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.543666 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.543641    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqlhs\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-kube-api-access-lqlhs\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.619870 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.619818    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.619958 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.619870    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.619958 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.619889    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620014    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44afef66-dcc8-4420-837d-2cd411e201cf-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620065    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620088    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620163 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620134    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620195 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620162    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620195 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620186    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sj26\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-kube-api-access-8sj26\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620280 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620267    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620315 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620290    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.620315 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.620308    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.676985 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.676962    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:12.720870 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.720838    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.720893    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.720914    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.720951    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44afef66-dcc8-4420-837d-2cd411e201cf-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.720973    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.720998    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721056    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721082    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721105    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sj26\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-kube-api-access-8sj26\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721148    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721183    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721262 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721213    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.721999 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721968    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.722130 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.721993    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.724246 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.724142    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.725275 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.724943    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.725275 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.725077    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.725275 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.725237    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.729013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.725574    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.729013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.728636    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.729013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.728690    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.729893 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.729851    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44afef66-dcc8-4420-837d-2cd411e201cf-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.730791 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.730769    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.749702 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.749677    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sj26\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-kube-api-access-8sj26\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:12.993317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:12.993291    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:32:12.994969 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:32:12.994942    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c56f84_462a_49a9_8bf8_d5bd3f43e278.slice/crio-d38a2a7e650d42590a4bfa4b78418a018e7a918a472803ab4fc6a7d956a0fb94 WatchSource:0}: Error finding container d38a2a7e650d42590a4bfa4b78418a018e7a918a472803ab4fc6a7d956a0fb94: Status 404 returned error can't find the container with id d38a2a7e650d42590a4bfa4b78418a018e7a918a472803ab4fc6a7d956a0fb94
Oct 30 11:32:13.634743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:13.634714    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerStarted","Data":"e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c"}
Oct 30 11:32:13.635121 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:13.634774    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerStarted","Data":"d38a2a7e650d42590a4bfa4b78418a018e7a918a472803ab4fc6a7d956a0fb94"}
Oct 30 11:32:17.657609 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:17.657573    4849 generic.go:356] "Generic (PLEG): container finished" podID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerID="e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c" exitCode=0
Oct 30 11:32:17.658016 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:17.657650    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerDied","Data":"e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c"}
Oct 30 11:32:18.672125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:18.672080    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerStarted","Data":"b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23"}
Oct 30 11:32:18.672510 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:18.672175    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerStarted","Data":"0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd"}
Oct 30 11:32:18.672510 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:18.672383    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:18.750703 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:18.750659    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podStartSLOduration=3.263857097 podStartE2EDuration="6.75064397s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.486786873s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:32:18.7361562 +0000 UTC m=+2836.116605655" watchObservedRunningTime="2026-10-30 11:32:18.75064397 +0000 UTC m=+2836.131093425"
Oct 30 11:32:22.089771 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.089726    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.121859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.121827    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.190974 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.190944    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.191134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.191067    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.207286 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.207266    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.311419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.311390    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5f4e377c3e259c071aab429e31ca61846b879cc3476e2f02b70e888ccdcdb590/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.339581 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.339560    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.386920 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.386869    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:22.723033 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:22.722998    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:32:22.725047 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:32:22.724999    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44afef66_dcc8_4420_837d_2cd411e201cf.slice/crio-7e80710093a249d56012e3e0c1828cca2763ad3394137980bc049af2a85b4888 WatchSource:0}: Error finding container 7e80710093a249d56012e3e0c1828cca2763ad3394137980bc049af2a85b4888: Status 404 returned error can't find the container with id 7e80710093a249d56012e3e0c1828cca2763ad3394137980bc049af2a85b4888
Oct 30 11:32:23.701355 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:23.701319    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerStarted","Data":"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"}
Oct 30 11:32:23.701779 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:23.701413    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerStarted","Data":"7e80710093a249d56012e3e0c1828cca2763ad3394137980bc049af2a85b4888"}
Oct 30 11:32:24.678655 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:24.678621    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:32:26.720087 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:26.720054    4849 generic.go:356] "Generic (PLEG): container finished" podID="44afef66-dcc8-4420-837d-2cd411e201cf" containerID="688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551" exitCode=0
Oct 30 11:32:26.720421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:26.720112    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerDied","Data":"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"}
Oct 30 11:32:27.730549 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:27.730517    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerStarted","Data":"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"}
Oct 30 11:32:27.730907 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:27.730581    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerStarted","Data":"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"}
Oct 30 11:32:27.730907 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:27.730606    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerStarted","Data":"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"}
Oct 30 11:32:27.814334 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:27.814290    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podStartSLOduration=12.328069965 podStartE2EDuration="15.814275781s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.486205816s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:32:27.799148341 +0000 UTC m=+2845.179597780" watchObservedRunningTime="2026-10-30 11:32:27.814275781 +0000 UTC m=+2845.194725653"
Oct 30 11:32:32.387878 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:32.387840    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:37.387654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:37.387613    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:37.393127 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:37.393096    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:37.788982 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:37.788901    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:32:39.430860 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:32:39.430826    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:32:46.390059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.389975    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:46.391054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.390295    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="df239dd2-c466-4e50-ac02-aef085e28ca2" containerName="otc-container" containerID="cri-o://2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d" gracePeriod=30
Oct 30 11:32:46.640267 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.640214    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:46.774663 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.774632    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-monitoring-tls-certs\") pod \"df239dd2-c466-4e50-ac02-aef085e28ca2\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") "
Oct 30 11:32:46.774825 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.774694    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-data-science-collector-ta-client-cert\") pod \"df239dd2-c466-4e50-ac02-aef085e28ca2\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") "
Oct 30 11:32:46.774825 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.774757    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-tls-certs\") pod \"df239dd2-c466-4e50-ac02-aef085e28ca2\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") "
Oct 30 11:32:46.774825 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.774776    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df239dd2-c466-4e50-ac02-aef085e28ca2-otc-internal\" (UniqueName: \"kubernetes.io/configmap/df239dd2-c466-4e50-ac02-aef085e28ca2-otc-internal\") pod \"df239dd2-c466-4e50-ac02-aef085e28ca2\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") "
Oct 30 11:32:46.774825 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.774812    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df239dd2-c466-4e50-ac02-aef085e28ca2-kube-api-access-9dpr5\" (UniqueName: \"kubernetes.io/projected/df239dd2-c466-4e50-ac02-aef085e28ca2-kube-api-access-9dpr5\") pod \"df239dd2-c466-4e50-ac02-aef085e28ca2\" (UID: \"df239dd2-c466-4e50-ac02-aef085e28ca2\") "
Oct 30 11:32:46.775251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.775205    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df239dd2-c466-4e50-ac02-aef085e28ca2-otc-internal" pod "df239dd2-c466-4e50-ac02-aef085e28ca2" (UID: "df239dd2-c466-4e50-ac02-aef085e28ca2"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:32:46.776924 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.776899    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-tls-certs" pod "df239dd2-c466-4e50-ac02-aef085e28ca2" (UID: "df239dd2-c466-4e50-ac02-aef085e28ca2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:32:46.777044 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.776968    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df239dd2-c466-4e50-ac02-aef085e28ca2-kube-api-access-9dpr5" pod "df239dd2-c466-4e50-ac02-aef085e28ca2" (UID: "df239dd2-c466-4e50-ac02-aef085e28ca2"). InnerVolumeSpecName "kube-api-access-9dpr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:32:46.777248 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.777227    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-data-science-collector-ta-client-cert" pod "df239dd2-c466-4e50-ac02-aef085e28ca2" (UID: "df239dd2-c466-4e50-ac02-aef085e28ca2"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:32:46.777296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.777264    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-monitoring-tls-certs" pod "df239dd2-c466-4e50-ac02-aef085e28ca2" (UID: "df239dd2-c466-4e50-ac02-aef085e28ca2"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:32:46.829306 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.829284    4849 generic.go:356] "Generic (PLEG): container finished" podID="df239dd2-c466-4e50-ac02-aef085e28ca2" containerID="2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d" exitCode=0
Oct 30 11:32:46.829413 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.829325    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:46.829413 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.829330    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"df239dd2-c466-4e50-ac02-aef085e28ca2","Type":"ContainerDied","Data":"2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d"}
Oct 30 11:32:46.829413 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.829368    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"df239dd2-c466-4e50-ac02-aef085e28ca2","Type":"ContainerDied","Data":"d5590a7b7fe568c0f419b15863db08f48afda2129cc64299f73095f056852d48"}
Oct 30 11:32:46.829413 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.829389    4849 scope.go:122] "RemoveContainer" containerID="2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d"
Oct 30 11:32:46.838465 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.838446    4849 scope.go:122] "RemoveContainer" containerID="2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d"
Oct 30 11:32:46.838726 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:32:46.838709    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d\": container with ID starting with 2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d not found: ID does not exist" containerID="2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d"
Oct 30 11:32:46.838771 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.838735    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d"} err="failed to get container status \"2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d\": rpc error: code = NotFound desc = could not find container \"2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d\": container with ID starting with 2715f378b70936021fad3cfe1a1c83b39bcb2ea27a1bcf88d06ba5cd8230191d not found: ID does not exist"
Oct 30 11:32:46.875994 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.875971    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:46.875994 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.875990    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/df239dd2-c466-4e50-ac02-aef085e28ca2-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:46.876162 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.875999    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dpr5\" (UniqueName: \"kubernetes.io/projected/df239dd2-c466-4e50-ac02-aef085e28ca2-kube-api-access-9dpr5\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:46.876162 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.876008    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:46.876162 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.876017    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/df239dd2-c466-4e50-ac02-aef085e28ca2-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:46.884270 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.884245    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:46.889054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.889017    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:46.927939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.926076    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:46.927939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.926588    4849 cpu_manager.go:414] "containerMap: removing container" podUID="df239dd2-c466-4e50-ac02-aef085e28ca2" containerName="otc-container"
Oct 30 11:32:46.927939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.926605    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df239dd2-c466-4e50-ac02-aef085e28ca2" containerName="otc-container"
Oct 30 11:32:46.927939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.926708    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="df239dd2-c466-4e50-ac02-aef085e28ca2" containerName="otc-container"
Oct 30 11:32:46.931390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.931370    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:46.948602 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.948581    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:32:46.948738 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.948728    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-2cltz\""
Oct 30 11:32:46.948931 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.948910    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:32:46.949065 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.948976    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:32:46.949287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.949263    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:32:46.950520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:46.950496    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:47.078181 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.078155    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.078313 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.078183    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/81d3e91c-4983-40d7-805f-fe29b033c9a1-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.078313 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.078257    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.078313 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.078298    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.078429 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.078319    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfdgg\" (UniqueName: \"kubernetes.io/projected/81d3e91c-4983-40d7-805f-fe29b033c9a1-kube-api-access-lfdgg\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.178980 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.178924    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.178980 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.178955    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfdgg\" (UniqueName: \"kubernetes.io/projected/81d3e91c-4983-40d7-805f-fe29b033c9a1-kube-api-access-lfdgg\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.179161 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.179012    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.179161 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.179059    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/81d3e91c-4983-40d7-805f-fe29b033c9a1-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.179161 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.179137    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.179743 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.179722    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/81d3e91c-4983-40d7-805f-fe29b033c9a1-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.181574 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.181555    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.181763 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.181743    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.181813 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.181743    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.203587 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.203567    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfdgg\" (UniqueName: \"kubernetes.io/projected/81d3e91c-4983-40d7-805f-fe29b033c9a1-kube-api-access-lfdgg\") pod \"data-science-collector-collector-0\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.243533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.243514    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:47.542884 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.542862    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:47.544609 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:32:47.544577    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d3e91c_4983_40d7_805f_fe29b033c9a1.slice/crio-a7e6bd62c96f81eceb6cdf3d21864aa2545396b07db50c6a3cb74a92e9128543 WatchSource:0}: Error finding container a7e6bd62c96f81eceb6cdf3d21864aa2545396b07db50c6a3cb74a92e9128543: Status 404 returned error can't find the container with id a7e6bd62c96f81eceb6cdf3d21864aa2545396b07db50c6a3cb74a92e9128543
Oct 30 11:32:47.546882 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.546853    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:32:47.546957 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.546906    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:32:47.836627 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.836593    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"81d3e91c-4983-40d7-805f-fe29b033c9a1","Type":"ContainerStarted","Data":"3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420"}
Oct 30 11:32:47.836796 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.836657    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"81d3e91c-4983-40d7-805f-fe29b033c9a1","Type":"ContainerStarted","Data":"a7e6bd62c96f81eceb6cdf3d21864aa2545396b07db50c6a3cb74a92e9128543"}
Oct 30 11:32:47.881534 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:47.881464    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.881439153 podStartE2EDuration="1.881439153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:32:47.880711009 +0000 UTC m=+2865.261160453" watchObservedRunningTime="2026-10-30 11:32:47.881439153 +0000 UTC m=+2865.261888594"
Oct 30 11:32:48.753120 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:48.753091    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df239dd2-c466-4e50-ac02-aef085e28ca2" path="/var/lib/kubelet/pods/df239dd2-c466-4e50-ac02-aef085e28ca2/volumes"
Oct 30 11:32:52.134590 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.134552    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:52.135132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.134782    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="81d3e91c-4983-40d7-805f-fe29b033c9a1" containerName="otc-container" containerID="cri-o://3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420" gracePeriod=30
Oct 30 11:32:52.373734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.373712    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:52.521054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.520969    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-data-science-collector-ta-client-cert\") pod \"81d3e91c-4983-40d7-805f-fe29b033c9a1\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") "
Oct 30 11:32:52.521174 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.521069    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81d3e91c-4983-40d7-805f-fe29b033c9a1-otc-internal\" (UniqueName: \"kubernetes.io/configmap/81d3e91c-4983-40d7-805f-fe29b033c9a1-otc-internal\") pod \"81d3e91c-4983-40d7-805f-fe29b033c9a1\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") "
Oct 30 11:32:52.521174 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.521101    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81d3e91c-4983-40d7-805f-fe29b033c9a1-kube-api-access-lfdgg\" (UniqueName: \"kubernetes.io/projected/81d3e91c-4983-40d7-805f-fe29b033c9a1-kube-api-access-lfdgg\") pod \"81d3e91c-4983-40d7-805f-fe29b033c9a1\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") "
Oct 30 11:32:52.521174 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.521169    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-monitoring-tls-certs\") pod \"81d3e91c-4983-40d7-805f-fe29b033c9a1\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") "
Oct 30 11:32:52.521354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.521231    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-tls-certs\") pod \"81d3e91c-4983-40d7-805f-fe29b033c9a1\" (UID: \"81d3e91c-4983-40d7-805f-fe29b033c9a1\") "
Oct 30 11:32:52.521507 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.521483    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81d3e91c-4983-40d7-805f-fe29b033c9a1-otc-internal" pod "81d3e91c-4983-40d7-805f-fe29b033c9a1" (UID: "81d3e91c-4983-40d7-805f-fe29b033c9a1"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:32:52.521612 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.521599    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/81d3e91c-4983-40d7-805f-fe29b033c9a1-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:52.523328 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.523296    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d3e91c-4983-40d7-805f-fe29b033c9a1-kube-api-access-lfdgg" pod "81d3e91c-4983-40d7-805f-fe29b033c9a1" (UID: "81d3e91c-4983-40d7-805f-fe29b033c9a1"). InnerVolumeSpecName "kube-api-access-lfdgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:32:52.523475 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.523450    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-tls-certs" pod "81d3e91c-4983-40d7-805f-fe29b033c9a1" (UID: "81d3e91c-4983-40d7-805f-fe29b033c9a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:32:52.523561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.523541    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-data-science-collector-ta-client-cert" pod "81d3e91c-4983-40d7-805f-fe29b033c9a1" (UID: "81d3e91c-4983-40d7-805f-fe29b033c9a1"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:32:52.523668 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.523648    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-monitoring-tls-certs" pod "81d3e91c-4983-40d7-805f-fe29b033c9a1" (UID: "81d3e91c-4983-40d7-805f-fe29b033c9a1"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:32:52.622398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.622368    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfdgg\" (UniqueName: \"kubernetes.io/projected/81d3e91c-4983-40d7-805f-fe29b033c9a1-kube-api-access-lfdgg\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:52.622501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.622401    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:52.622501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.622431    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:52.622501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.622448    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/81d3e91c-4983-40d7-805f-fe29b033c9a1-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:32:52.864732 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.864696    4849 generic.go:356] "Generic (PLEG): container finished" podID="81d3e91c-4983-40d7-805f-fe29b033c9a1" containerID="3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420" exitCode=0
Oct 30 11:32:52.864908 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.864751    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:52.864908 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.864800    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"81d3e91c-4983-40d7-805f-fe29b033c9a1","Type":"ContainerDied","Data":"3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420"}
Oct 30 11:32:52.864908 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.864865    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"81d3e91c-4983-40d7-805f-fe29b033c9a1","Type":"ContainerDied","Data":"a7e6bd62c96f81eceb6cdf3d21864aa2545396b07db50c6a3cb74a92e9128543"}
Oct 30 11:32:52.864908 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.864898    4849 scope.go:122] "RemoveContainer" containerID="3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420"
Oct 30 11:32:52.873624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.873607    4849 scope.go:122] "RemoveContainer" containerID="3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420"
Oct 30 11:32:52.873874 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:32:52.873854    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420\": container with ID starting with 3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420 not found: ID does not exist" containerID="3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420"
Oct 30 11:32:52.873921 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.873883    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420"} err="failed to get container status \"3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420\": rpc error: code = NotFound desc = could not find container \"3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420\": container with ID starting with 3ed0096529054c7f4fa98bdff80397ef019add4806212d7acf0f330768e3c420 not found: ID does not exist"
Oct 30 11:32:52.919904 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.919880    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:52.933288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.933267    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:52.960104 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.960081    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:52.960456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.960440    4849 cpu_manager.go:414] "containerMap: removing container" podUID="81d3e91c-4983-40d7-805f-fe29b033c9a1" containerName="otc-container"
Oct 30 11:32:52.960507 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.960459    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81d3e91c-4983-40d7-805f-fe29b033c9a1" containerName="otc-container"
Oct 30 11:32:52.960545 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.960531    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="81d3e91c-4983-40d7-805f-fe29b033c9a1" containerName="otc-container"
Oct 30 11:32:52.966942 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.966917    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:52.985016 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.984996    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:52.986505 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.986484    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:32:52.986607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.986534    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:32:52.986607 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.986544    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:32:52.986716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.986607    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-2cltz\""
Oct 30 11:32:52.987892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:52.987875    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:32:53.126432 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.126368    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.126432 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.126411    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.126608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.126448    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.126608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.126485    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.126608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.126514    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6mjb\" (UniqueName: \"kubernetes.io/projected/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-kube-api-access-q6mjb\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.226892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.226867    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.227264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.226898    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.227264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.226919    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.227264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.227050    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.227264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.227093    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6mjb\" (UniqueName: \"kubernetes.io/projected/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-kube-api-access-q6mjb\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.227753 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.227732    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.229254 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.229230    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.229375 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.229357    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.229447 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.229396    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.251400 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.251375    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6mjb\" (UniqueName: \"kubernetes.io/projected/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-kube-api-access-q6mjb\") pod \"data-science-collector-collector-0\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.280794 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.280774    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:32:53.579284 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.579258    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:32:53.581236 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:32:53.581207    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9983db9_e5d0_4b5a_84c1_6fa21a704d2b.slice/crio-fe5f14749e69c917587d5892c91c810abcbe5ec9a6eb1487fdb78fceb5e08d74 WatchSource:0}: Error finding container fe5f14749e69c917587d5892c91c810abcbe5ec9a6eb1487fdb78fceb5e08d74: Status 404 returned error can't find the container with id fe5f14749e69c917587d5892c91c810abcbe5ec9a6eb1487fdb78fceb5e08d74
Oct 30 11:32:53.583255 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.583224    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:32:53.583340 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.583281    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:32:53.872285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.872204    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b","Type":"ContainerStarted","Data":"2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8"}
Oct 30 11:32:53.872285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.872266    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b","Type":"ContainerStarted","Data":"fe5f14749e69c917587d5892c91c810abcbe5ec9a6eb1487fdb78fceb5e08d74"}
Oct 30 11:32:53.917692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:53.917646    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.917632519 podStartE2EDuration="1.917632519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:32:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:32:53.916664014 +0000 UTC m=+2871.297113453" watchObservedRunningTime="2026-10-30 11:32:53.917632519 +0000 UTC m=+2871.298081958"
Oct 30 11:32:54.754113 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:32:54.754076    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81d3e91c-4983-40d7-805f-fe29b033c9a1" path="/var/lib/kubelet/pods/81d3e91c-4983-40d7-805f-fe29b033c9a1/volumes"
Oct 30 11:33:09.432966 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:09.432925    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:33:19.066537 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.066500    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:33:19.067150 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.066813    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="alertmanager" containerID="cri-o://0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd" gracePeriod=120
Oct 30 11:33:19.067150 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.066847    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="config-reloader" containerID="cri-o://b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23" gracePeriod=120
Oct 30 11:33:19.080490 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.080457    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:33:19.080773 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.080744    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="prometheus" containerID="cri-o://c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0" gracePeriod=600
Oct 30 11:33:19.080993 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.080783    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="config-reloader" containerID="cri-o://11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468" gracePeriod=600
Oct 30 11:33:19.081108 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.080837    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="thanos-sidecar" containerID="cri-o://342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400" gracePeriod=600
Oct 30 11:33:19.200187 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:19.200145    4849 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44afef66_dcc8_4420_837d_2cd411e201cf.slice/crio-conmon-11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c56f84_462a_49a9_8bf8_d5bd3f43e278.slice/crio-b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 11:33:19.254638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.254601    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:19.255218 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.255125    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" containerName="otc-container" containerID="cri-o://2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8" gracePeriod=30
Oct 30 11:33:19.371198 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.371174    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:33:19.429411 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429374    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429474    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-config\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429525    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429556    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-kube-api-access-8sj26\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-kube-api-access-8sj26\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429761 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429595    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429761 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429660    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-thanos-prometheus-http-client-file\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429761 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429707    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-tls-assets\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-tls-assets\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429761 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429741    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44afef66-dcc8-4420-837d-2cd411e201cf-config-out\" (UniqueName: \"kubernetes.io/empty-dir/44afef66-dcc8-4420-837d-2cd411e201cf-config-out\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429803    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429831    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-secret-prometheus-web-tls-ca\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.429950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429858    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.430148 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.429974    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-2" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:33:19.430206 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.430181    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.430712 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.430683    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-1" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:33:19.433072 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.433007    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.434153 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.434114    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-tls-assets" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:33:19.434379 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.434339    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-secret-prometheus-web-tls-ca" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.435098 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.435033    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-kube-api-access-8sj26" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "kube-api-access-8sj26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:33:19.435205 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.435138    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-thanos-prometheus-http-client-file" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.437286 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.437205    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44afef66-dcc8-4420-837d-2cd411e201cf-config-out" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:33:19.437404 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.437333    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.437702 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.437681    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-config" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.450740 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.450714    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.522215 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.522189    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:19.531087 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531065    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-data-science-collector-ta-client-cert\") pod \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") "
Oct 30 11:33:19.531195 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531125    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-tls-certs\") pod \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") "
Oct 30 11:33:19.531195 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531142    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-monitoring-tls-certs\") pod \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") "
Oct 30 11:33:19.531195 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531164    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.531363 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531230    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"44afef66-dcc8-4420-837d-2cd411e201cf\" (UID: \"44afef66-dcc8-4420-837d-2cd411e201cf\") "
Oct 30 11:33:19.531363 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531251    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-kube-api-access-q6mjb\" (UniqueName: \"kubernetes.io/projected/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-kube-api-access-q6mjb\") pod \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") "
Oct 30 11:33:19.531363 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531282    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-otc-internal\" (UniqueName: \"kubernetes.io/configmap/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-otc-internal\") pod \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\" (UID: \"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b\") "
Oct 30 11:33:19.531560 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531531    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531560 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531558    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531575    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sj26\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-kube-api-access-8sj26\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531595    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531612    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44afef66-dcc8-4420-837d-2cd411e201cf-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531626    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44afef66-dcc8-4420-837d-2cd411e201cf-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531640    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531655    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531651    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-0" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531669    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.531692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531684    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44afef66-dcc8-4420-837d-2cd411e201cf-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.532210 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.531985    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-otc-internal" pod "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" (UID: "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:33:19.533286 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.533261    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-monitoring-tls-certs" pod "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" (UID: "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.533384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.533364    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-tls-certs" pod "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" (UID: "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.533491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.533460    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-kube-api-access-q6mjb" pod "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" (UID: "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b"). InnerVolumeSpecName "kube-api-access-q6mjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:33:19.533583 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.533553    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-data-science-collector-ta-client-cert" pod "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" (UID: "b9983db9-e5d0-4b5a-84c1-6fa21a704d2b"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:19.535835 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.535817    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396" pod "44afef66-dcc8-4420-837d-2cd411e201cf" (UID: "44afef66-dcc8-4420-837d-2cd411e201cf"). InnerVolumeSpecName "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:33:19.632118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.632059    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.632118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.632083    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.632118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.632094    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.632118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.632103    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44afef66-dcc8-4420-837d-2cd411e201cf-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.632318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.632129    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:33:19.632318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.632140    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6mjb\" (UniqueName: \"kubernetes.io/projected/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-kube-api-access-q6mjb\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.632318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.632148    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:19.643685 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.643670    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:33:19.678050 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.678011    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.111:9093/-/ready\": dial tcp 10.132.0.111:9093: connect: connection refused"
Oct 30 11:33:19.732459 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:19.732434    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\""
Oct 30 11:33:20.003267 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003205    4849 generic.go:356] "Generic (PLEG): container finished" podID="44afef66-dcc8-4420-837d-2cd411e201cf" containerID="342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400" exitCode=0
Oct 30 11:33:20.003267 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003227    4849 generic.go:356] "Generic (PLEG): container finished" podID="44afef66-dcc8-4420-837d-2cd411e201cf" containerID="11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468" exitCode=0
Oct 30 11:33:20.003267 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003237    4849 generic.go:356] "Generic (PLEG): container finished" podID="44afef66-dcc8-4420-837d-2cd411e201cf" containerID="c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0" exitCode=0
Oct 30 11:33:20.003486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003278    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:33:20.003486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003288    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerDied","Data":"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"}
Oct 30 11:33:20.003486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003351    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerDied","Data":"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"}
Oct 30 11:33:20.003486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003376    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerDied","Data":"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"}
Oct 30 11:33:20.003486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003409    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"44afef66-dcc8-4420-837d-2cd411e201cf","Type":"ContainerDied","Data":"7e80710093a249d56012e3e0c1828cca2763ad3394137980bc049af2a85b4888"}
Oct 30 11:33:20.003486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.003425    4849 scope.go:122] "RemoveContainer" containerID="342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"
Oct 30 11:33:20.006076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.006052    4849 generic.go:356] "Generic (PLEG): container finished" podID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerID="b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23" exitCode=0
Oct 30 11:33:20.006160 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.006114    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerDied","Data":"b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23"}
Oct 30 11:33:20.007665 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.007645    4849 generic.go:356] "Generic (PLEG): container finished" podID="b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" containerID="2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8" exitCode=0
Oct 30 11:33:20.007759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.007686    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:20.007759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.007723    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b","Type":"ContainerDied","Data":"2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8"}
Oct 30 11:33:20.007759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.007757    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"b9983db9-e5d0-4b5a-84c1-6fa21a704d2b","Type":"ContainerDied","Data":"fe5f14749e69c917587d5892c91c810abcbe5ec9a6eb1487fdb78fceb5e08d74"}
Oct 30 11:33:20.012848 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.012827    4849 scope.go:122] "RemoveContainer" containerID="11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"
Oct 30 11:33:20.021928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.021911    4849 scope.go:122] "RemoveContainer" containerID="c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"
Oct 30 11:33:20.029794 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.029776    4849 scope.go:122] "RemoveContainer" containerID="688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"
Oct 30 11:33:20.037294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.037276    4849 scope.go:122] "RemoveContainer" containerID="342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"
Oct 30 11:33:20.037574 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:20.037556    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400\": container with ID starting with 342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400 not found: ID does not exist" containerID="342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"
Oct 30 11:33:20.037626 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.037581    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"} err="failed to get container status \"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400\": rpc error: code = NotFound desc = could not find container \"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400\": container with ID starting with 342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400 not found: ID does not exist"
Oct 30 11:33:20.037626 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.037600    4849 scope.go:122] "RemoveContainer" containerID="11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"
Oct 30 11:33:20.037849 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:20.037827    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468\": container with ID starting with 11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468 not found: ID does not exist" containerID="11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"
Oct 30 11:33:20.037902 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.037857    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"} err="failed to get container status \"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468\": rpc error: code = NotFound desc = could not find container \"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468\": container with ID starting with 11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468 not found: ID does not exist"
Oct 30 11:33:20.037902 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.037877    4849 scope.go:122] "RemoveContainer" containerID="c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"
Oct 30 11:33:20.038145 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:20.038126    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0\": container with ID starting with c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0 not found: ID does not exist" containerID="c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"
Oct 30 11:33:20.038224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038151    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"} err="failed to get container status \"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0\": rpc error: code = NotFound desc = could not find container \"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0\": container with ID starting with c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0 not found: ID does not exist"
Oct 30 11:33:20.038224 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038170    4849 scope.go:122] "RemoveContainer" containerID="688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"
Oct 30 11:33:20.038402 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:20.038387    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551\": container with ID starting with 688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551 not found: ID does not exist" containerID="688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"
Oct 30 11:33:20.038467 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038408    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"} err="failed to get container status \"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551\": rpc error: code = NotFound desc = could not find container \"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551\": container with ID starting with 688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551 not found: ID does not exist"
Oct 30 11:33:20.038467 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038429    4849 scope.go:122] "RemoveContainer" containerID="342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"
Oct 30 11:33:20.038649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038631    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"} err="failed to get container status \"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400\": rpc error: code = NotFound desc = could not find container \"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400\": container with ID starting with 342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400 not found: ID does not exist"
Oct 30 11:33:20.038649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038649    4849 scope.go:122] "RemoveContainer" containerID="11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"
Oct 30 11:33:20.038843 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038826    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"} err="failed to get container status \"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468\": rpc error: code = NotFound desc = could not find container \"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468\": container with ID starting with 11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468 not found: ID does not exist"
Oct 30 11:33:20.038888 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.038842    4849 scope.go:122] "RemoveContainer" containerID="c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"
Oct 30 11:33:20.039040 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039011    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"} err="failed to get container status \"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0\": rpc error: code = NotFound desc = could not find container \"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0\": container with ID starting with c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0 not found: ID does not exist"
Oct 30 11:33:20.039099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039042    4849 scope.go:122] "RemoveContainer" containerID="688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"
Oct 30 11:33:20.039222 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039208    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"} err="failed to get container status \"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551\": rpc error: code = NotFound desc = could not find container \"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551\": container with ID starting with 688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551 not found: ID does not exist"
Oct 30 11:33:20.039222 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039221    4849 scope.go:122] "RemoveContainer" containerID="342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"
Oct 30 11:33:20.039429 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039407    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400"} err="failed to get container status \"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400\": rpc error: code = NotFound desc = could not find container \"342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400\": container with ID starting with 342e236f5e7b4fa832b870f3a7955d454dbe339eed4b07da061eaf00bd777400 not found: ID does not exist"
Oct 30 11:33:20.039429 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039428    4849 scope.go:122] "RemoveContainer" containerID="11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"
Oct 30 11:33:20.039660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039644    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468"} err="failed to get container status \"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468\": rpc error: code = NotFound desc = could not find container \"11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468\": container with ID starting with 11ac6173d9793568440915c5454567e2154060bfb59c104aa832990f2d010468 not found: ID does not exist"
Oct 30 11:33:20.039706 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039660    4849 scope.go:122] "RemoveContainer" containerID="c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"
Oct 30 11:33:20.039863 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039848    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0"} err="failed to get container status \"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0\": rpc error: code = NotFound desc = could not find container \"c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0\": container with ID starting with c4006e4f4b923f980fce74f47a9eb4f219bb05dd36ecd988695c2c897c631fd0 not found: ID does not exist"
Oct 30 11:33:20.039863 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.039862    4849 scope.go:122] "RemoveContainer" containerID="688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"
Oct 30 11:33:20.040113 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.040094    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551"} err="failed to get container status \"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551\": rpc error: code = NotFound desc = could not find container \"688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551\": container with ID starting with 688b7af14640d85de21fde8f678c40044f8788aafed2c67543345cf925b2f551 not found: ID does not exist"
Oct 30 11:33:20.040169 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.040116    4849 scope.go:122] "RemoveContainer" containerID="2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8"
Oct 30 11:33:20.047202 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.047183    4849 scope.go:122] "RemoveContainer" containerID="2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8"
Oct 30 11:33:20.047419 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:20.047401    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8\": container with ID starting with 2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8 not found: ID does not exist" containerID="2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8"
Oct 30 11:33:20.047469 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.047424    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8"} err="failed to get container status \"2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8\": rpc error: code = NotFound desc = could not find container \"2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8\": container with ID starting with 2444de800978f93fafb4818f64ec22a453d661afa77238022257e26511c9f0d8 not found: ID does not exist"
Oct 30 11:33:20.061670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.061650    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:33:20.075277 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.075251    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:33:20.102239 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.102220    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:20.106556 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.106537    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:20.753665 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.753634    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" path="/var/lib/kubelet/pods/44afef66-dcc8-4420-837d-2cd411e201cf/volumes"
Oct 30 11:33:20.754291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:20.754270    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" path="/var/lib/kubelet/pods/b9983db9-e5d0-4b5a-84c1-6fa21a704d2b/volumes"
Oct 30 11:33:24.678233 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:24.678189    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.111:9093/-/ready\": dial tcp 10.132.0.111:9093: connect: connection refused"
Oct 30 11:33:27.910475 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910439    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910791    4849 cpu_manager.go:414] "containerMap: removing container" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="init-config-reloader"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910802    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="init-config-reloader"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910813    4849 cpu_manager.go:414] "containerMap: removing container" podUID="b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" containerName="otc-container"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910818    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" containerName="otc-container"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910845    4849 cpu_manager.go:414] "containerMap: removing container" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="prometheus"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910851    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="prometheus"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910857    4849 cpu_manager.go:414] "containerMap: removing container" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="thanos-sidecar"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910863    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="thanos-sidecar"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910874    4849 cpu_manager.go:414] "containerMap: removing container" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="config-reloader"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910881    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="config-reloader"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910945    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="config-reloader"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910957    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9983db9-e5d0-4b5a-84c1-6fa21a704d2b" containerName="otc-container"
Oct 30 11:33:27.910959 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910968    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="prometheus"
Oct 30 11:33:27.911669 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.910974    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="44afef66-dcc8-4420-837d-2cd411e201cf" containerName="thanos-sidecar"
Oct 30 11:33:27.915987 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.915967    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:27.944686 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.944659    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:27.955840 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.955815    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:33:27.955999 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.955937    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:33:27.956139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.955940    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-dr7t6\""
Oct 30 11:33:27.956139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.955957    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:33:27.956139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.955982    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:33:27.992634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.992602    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/42de0203-4d99-4a4a-8fff-576a0da8fd36-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:27.992802 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.992652    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:27.992802 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.992683    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:27.992802 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.992719    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:27.992802 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:27.992760    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4x6t\" (UniqueName: \"kubernetes.io/projected/42de0203-4d99-4a4a-8fff-576a0da8fd36-kube-api-access-k4x6t\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.094219 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.094182    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/42de0203-4d99-4a4a-8fff-576a0da8fd36-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.094219 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.094225    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.094443 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.094246    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.094443 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.094262    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.094443 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.094289    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4x6t\" (UniqueName: \"kubernetes.io/projected/42de0203-4d99-4a4a-8fff-576a0da8fd36-kube-api-access-k4x6t\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.094931 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.094904    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/42de0203-4d99-4a4a-8fff-576a0da8fd36-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.096679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.096652    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.096782 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.096690    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.097055 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.097038    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.122129 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.122104    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4x6t\" (UniqueName: \"kubernetes.io/projected/42de0203-4d99-4a4a-8fff-576a0da8fd36-kube-api-access-k4x6t\") pod \"data-science-collector-collector-0\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.230227 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.230171    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:28.529980 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.529959    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:28.531840 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:33:28.531811    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42de0203_4d99_4a4a_8fff_576a0da8fd36.slice/crio-8c11222c6b1ce3c3599424f35264fab76ba829c18bf65f64c0b43e1168d11988 WatchSource:0}: Error finding container 8c11222c6b1ce3c3599424f35264fab76ba829c18bf65f64c0b43e1168d11988: Status 404 returned error can't find the container with id 8c11222c6b1ce3c3599424f35264fab76ba829c18bf65f64c0b43e1168d11988
Oct 30 11:33:28.533851 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.533820    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:33:28.533919 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:28.533899    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:33:29.063449 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.063415    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"42de0203-4d99-4a4a-8fff-576a0da8fd36","Type":"ContainerStarted","Data":"53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955"}
Oct 30 11:33:29.063829 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.063467    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"42de0203-4d99-4a4a-8fff-576a0da8fd36","Type":"ContainerStarted","Data":"8c11222c6b1ce3c3599424f35264fab76ba829c18bf65f64c0b43e1168d11988"}
Oct 30 11:33:29.108939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.108899    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=2.10888456 podStartE2EDuration="2.10888456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:33:29.10704659 +0000 UTC m=+2906.487496029" watchObservedRunningTime="2026-10-30 11:33:29.10888456 +0000 UTC m=+2906.489333998"
Oct 30 11:33:29.320566 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.320510    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:29.404296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.404265    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-web-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-web-config\") pod \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") "
Oct 30 11:33:29.404612 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.404578    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-tls-assets\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-tls-assets\") pod \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") "
Oct 30 11:33:29.404726 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.404653    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-out\") pod \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") "
Oct 30 11:33:29.404726 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.404715    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-kube-api-access-lqlhs\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-kube-api-access-lqlhs\") pod \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") "
Oct 30 11:33:29.404849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.404761    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-alertmanager-data-science-monitoringstack-db\") pod \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") "
Oct 30 11:33:29.404849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.404798    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-cluster-tls-config\") pod \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") "
Oct 30 11:33:29.404849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.404828    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-volume\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-volume\") pod \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\" (UID: \"a2c56f84-462a-49a9-8bf8-d5bd3f43e278\") "
Oct 30 11:33:29.405278 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.405245    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-alertmanager-data-science-monitoringstack-db" pod "a2c56f84-462a-49a9-8bf8-d5bd3f43e278" (UID: "a2c56f84-462a-49a9-8bf8-d5bd3f43e278"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:33:29.407620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.407581    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-kube-api-access-lqlhs" pod "a2c56f84-462a-49a9-8bf8-d5bd3f43e278" (UID: "a2c56f84-462a-49a9-8bf8-d5bd3f43e278"). InnerVolumeSpecName "kube-api-access-lqlhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:33:29.407620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.407598    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-volume" pod "a2c56f84-462a-49a9-8bf8-d5bd3f43e278" (UID: "a2c56f84-462a-49a9-8bf8-d5bd3f43e278"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:29.407770 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.407598    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-tls-assets" pod "a2c56f84-462a-49a9-8bf8-d5bd3f43e278" (UID: "a2c56f84-462a-49a9-8bf8-d5bd3f43e278"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:33:29.408251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.408227    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-out" pod "a2c56f84-462a-49a9-8bf8-d5bd3f43e278" (UID: "a2c56f84-462a-49a9-8bf8-d5bd3f43e278"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:33:29.411757 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.411731    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-cluster-tls-config" pod "a2c56f84-462a-49a9-8bf8-d5bd3f43e278" (UID: "a2c56f84-462a-49a9-8bf8-d5bd3f43e278"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:29.418972 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.418948    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-web-config" pod "a2c56f84-462a-49a9-8bf8-d5bd3f43e278" (UID: "a2c56f84-462a-49a9-8bf8-d5bd3f43e278"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:29.506371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.506347    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:29.506371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.506368    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:29.506498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.506377    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:29.506498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.506385    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqlhs\" (UniqueName: \"kubernetes.io/projected/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-kube-api-access-lqlhs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:29.506498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.506394    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:29.506498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.506403    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:29.506498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:29.506410    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2c56f84-462a-49a9-8bf8-d5bd3f43e278-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:30.070883 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.070852    4849 generic.go:356] "Generic (PLEG): container finished" podID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerID="0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd" exitCode=0
Oct 30 11:33:30.071371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.070926    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:30.071371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.070936    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerDied","Data":"0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd"}
Oct 30 11:33:30.071371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.071005    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"a2c56f84-462a-49a9-8bf8-d5bd3f43e278","Type":"ContainerDied","Data":"d38a2a7e650d42590a4bfa4b78418a018e7a918a472803ab4fc6a7d956a0fb94"}
Oct 30 11:33:30.071371 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.071045    4849 scope.go:122] "RemoveContainer" containerID="b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23"
Oct 30 11:33:30.081236 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.081219    4849 scope.go:122] "RemoveContainer" containerID="0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd"
Oct 30 11:33:30.090621 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.090404    4849 scope.go:122] "RemoveContainer" containerID="e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c"
Oct 30 11:33:30.108249 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.108227    4849 scope.go:122] "RemoveContainer" containerID="b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23"
Oct 30 11:33:30.108577 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:30.108553    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23\": container with ID starting with b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23 not found: ID does not exist" containerID="b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23"
Oct 30 11:33:30.108644 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.108582    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23"} err="failed to get container status \"b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23\": rpc error: code = NotFound desc = could not find container \"b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23\": container with ID starting with b33e4621ad911dd920c4e9d8382bafd84064940c6ed587a755372f402e0c4b23 not found: ID does not exist"
Oct 30 11:33:30.108644 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.108598    4849 scope.go:122] "RemoveContainer" containerID="0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd"
Oct 30 11:33:30.108889 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:30.108872    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd\": container with ID starting with 0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd not found: ID does not exist" containerID="0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd"
Oct 30 11:33:30.108956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.108892    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd"} err="failed to get container status \"0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd\": rpc error: code = NotFound desc = could not find container \"0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd\": container with ID starting with 0862f254a085bb779d7f15c105864d2e1bafa388fe43f0ef2a167053fd17ffbd not found: ID does not exist"
Oct 30 11:33:30.108956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.108913    4849 scope.go:122] "RemoveContainer" containerID="e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c"
Oct 30 11:33:30.109204 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:30.109183    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c\": container with ID starting with e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c not found: ID does not exist" containerID="e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c"
Oct 30 11:33:30.109261 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.109209    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c"} err="failed to get container status \"e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c\": rpc error: code = NotFound desc = could not find container \"e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c\": container with ID starting with e332514cd4a64f1bae1dba5838c3aa3dae1083aee29bd950dd81b99dea46185c not found: ID does not exist"
Oct 30 11:33:30.131899 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.131877    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:33:30.134931 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.134909    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:33:30.755377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:30.755340    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" path="/var/lib/kubelet/pods/a2c56f84-462a-49a9-8bf8-d5bd3f43e278/volumes"
Oct 30 11:33:31.322816 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.322778    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:33:31.323200 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323165    4849 cpu_manager.go:414] "containerMap: removing container" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="config-reloader"
Oct 30 11:33:31.323200 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323177    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="config-reloader"
Oct 30 11:33:31.323200 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323194    4849 cpu_manager.go:414] "containerMap: removing container" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="alertmanager"
Oct 30 11:33:31.323200 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323199    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="alertmanager"
Oct 30 11:33:31.323329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323220    4849 cpu_manager.go:414] "containerMap: removing container" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="init-config-reloader"
Oct 30 11:33:31.323329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323225    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="init-config-reloader"
Oct 30 11:33:31.323329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323297    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="config-reloader"
Oct 30 11:33:31.323329 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.323309    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2c56f84-462a-49a9-8bf8-d5bd3f43e278" containerName="alertmanager"
Oct 30 11:33:31.328237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.328212    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.348160 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.348138    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:33:31.348309 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.348193    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 11:33:31.348380 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.348304    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 11:33:31.348380 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.348213    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 11:33:31.348380 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.348353    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-c9rbj\""
Oct 30 11:33:31.363406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.363383    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:33:31.386118 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.386090    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:33:31.390178 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.390156    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.421248 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.421224    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:33:31.448579 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.448555    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 11:33:31.448579 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.448561    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-gg924\""
Oct 30 11:33:31.448736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.448580    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:33:31.448736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.448618    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 11:33:31.448736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.448636    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 11:33:31.448736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.448627    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 11:33:31.448736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.448645    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 11:33:31.485769 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.485749    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 11:33:31.485858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.485795    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 11:33:31.486047 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.486009    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 11:33:31.522318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522297    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-tls-assets\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522330    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522394 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522350    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522464 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522424    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522460    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.522498 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522485    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522604 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522589    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.522638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522612    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.522638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522626    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.522694 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522654    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522754 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522737    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gqhp\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-kube-api-access-2gqhp\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522792 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522768    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522825 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522790    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1026b4d2-2d29-4321-9e85-5489db9ac13a-config-out\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522825 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522808    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522905 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522845    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.522948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522906    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.522948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522933    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq8wr\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-kube-api-access-tq8wr\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.523015 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.522977    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.523015 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.523010    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.624003 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.623939    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624003 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.623981    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624006    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624055    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.624193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624086    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624121    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.624193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624143    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.624193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624167    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624204    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624235    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gqhp\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-kube-api-access-2gqhp\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624269    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624301    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1026b4d2-2d29-4321-9e85-5489db9ac13a-config-out\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624327    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624357    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624393    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624419    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq8wr\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-kube-api-access-tq8wr\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624452    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.624506 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624503    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.625048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.624561    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-tls-assets\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.625783 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.625756    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.627274 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.627091    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.627453 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.627403    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.627547 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.627498    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-tls-assets\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.627621 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.627561    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.627963 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.627939    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.628084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.628008    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.628366 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.628343    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.628451 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.628414    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.628684 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.628657    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.629399 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.629372    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.629505 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.629403    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.629641 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.629620    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1026b4d2-2d29-4321-9e85-5489db9ac13a-config-out\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.629977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.629957    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.630112 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.630076    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.630369 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.630348    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.630859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.630839    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.650809 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.650783    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gqhp\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-kube-api-access-2gqhp\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:31.650946 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.650923    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq8wr\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-kube-api-access-tq8wr\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:31.946455 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:31.946393    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:32.280976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:32.280958    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:33:32.283070 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:33:32.283037    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod656ad963_6770_4df3_bd17_005b40732b69.slice/crio-7f42b2bc0e870f5974f7b0f1cbca2f38ecc47e20d6722750261f6cbbd7d3085a WatchSource:0}: Error finding container 7f42b2bc0e870f5974f7b0f1cbca2f38ecc47e20d6722750261f6cbbd7d3085a: Status 404 returned error can't find the container with id 7f42b2bc0e870f5974f7b0f1cbca2f38ecc47e20d6722750261f6cbbd7d3085a
Oct 30 11:33:33.089961 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:33.089929    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerStarted","Data":"e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509"}
Oct 30 11:33:33.090295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:33.089988    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerStarted","Data":"7f42b2bc0e870f5974f7b0f1cbca2f38ecc47e20d6722750261f6cbbd7d3085a"}
Oct 30 11:33:34.344724 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.344692    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:34.375737 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.375708    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:34.445310 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.445281    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:34.445418 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.445370    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:34.460682 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.460656    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") DevicePath \"csi-bc5b0eea9a26be0763bd2234369df1d0cb0bca5610c96b98228e00775a514ae6\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:34.561486 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.561460    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/7342da0537aa528cc1545d2018ac3fd0382dfe4f670067e58cf8ba34bb891174/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:34.589475 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.589452    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"prometheus-data-science-monitoringstack-1\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:34.714335 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:34.714281    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:35.048456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:35.048435    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:33:35.049996 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:33:35.049961    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1026b4d2_2d29_4321_9e85_5489db9ac13a.slice/crio-5143d51bfcc27a6b905b93c7f4a970bd90239857364a9f1b208f13e007219608 WatchSource:0}: Error finding container 5143d51bfcc27a6b905b93c7f4a970bd90239857364a9f1b208f13e007219608: Status 404 returned error can't find the container with id 5143d51bfcc27a6b905b93c7f4a970bd90239857364a9f1b208f13e007219608
Oct 30 11:33:35.101354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:35.101330    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerStarted","Data":"5143d51bfcc27a6b905b93c7f4a970bd90239857364a9f1b208f13e007219608"}
Oct 30 11:33:36.108515 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:36.108473    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerStarted","Data":"9f122db444f1bc4380944fed9c6c0d3b529a9181d6d90985ad9857ce073146b1"}
Oct 30 11:33:36.110037 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:36.109999    4849 generic.go:356] "Generic (PLEG): container finished" podID="656ad963-6770-4df3-bd17-005b40732b69" containerID="e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509" exitCode=0
Oct 30 11:33:36.110134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:36.110072    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerDied","Data":"e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509"}
Oct 30 11:33:37.117032 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:37.116998    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerStarted","Data":"ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc"}
Oct 30 11:33:37.117391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:37.117073    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerStarted","Data":"1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222"}
Oct 30 11:33:37.117465 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:37.117443    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:37.196406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:37.196358    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podStartSLOduration=2.8117707039999997 podStartE2EDuration="6.196345337s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.384574633s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:33:37.181121323 +0000 UTC m=+2914.561570763" watchObservedRunningTime="2026-10-30 11:33:37.196345337 +0000 UTC m=+2914.576794777"
Oct 30 11:33:39.129134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:39.129102    4849 generic.go:356] "Generic (PLEG): container finished" podID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerID="9f122db444f1bc4380944fed9c6c0d3b529a9181d6d90985ad9857ce073146b1" exitCode=0
Oct 30 11:33:39.129536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:39.129177    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerDied","Data":"9f122db444f1bc4380944fed9c6c0d3b529a9181d6d90985ad9857ce073146b1"}
Oct 30 11:33:39.431431 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:39.431384    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:33:40.137145 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:40.137112    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerStarted","Data":"cff2844bdfa862e991dfdeaba342920020fd80708cafdf670eb5aa89f0ea1b61"}
Oct 30 11:33:40.137529 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:40.137205    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerStarted","Data":"de9db7a8b51450df818f5623dea34d77dd08e316a869dd2b4c0cc4801fe8b8ec"}
Oct 30 11:33:40.137529 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:40.137232    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerStarted","Data":"3925665e4f1d7bb6143259e64973cd40053a63d043e71e5dbbbc20b6c1965499"}
Oct 30 11:33:40.220894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:40.220842    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podStartSLOduration=5.83528931 podStartE2EDuration="9.220829567s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.385540257s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:33:40.20587021 +0000 UTC m=+2917.586319649" watchObservedRunningTime="2026-10-30 11:33:40.220829567 +0000 UTC m=+2917.601279005"
Oct 30 11:33:43.124128 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:43.124095    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:33:44.714846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:44.714758    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:48.755780 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:48.755738    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:48.756631 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:48.756016    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="42de0203-4d99-4a4a-8fff-576a0da8fd36" containerName="otc-container" containerID="cri-o://53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955" gracePeriod=30
Oct 30 11:33:49.715038 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:49.714992    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:49.720364 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:49.720348    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:50.197314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:50.197286    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:33:51.112380 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.112357    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.198357 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.198283    4849 generic.go:356] "Generic (PLEG): container finished" podID="42de0203-4d99-4a4a-8fff-576a0da8fd36" containerID="53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955" exitCode=0
Oct 30 11:33:51.198357 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.198333    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.198790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.198368    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"42de0203-4d99-4a4a-8fff-576a0da8fd36","Type":"ContainerDied","Data":"53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955"}
Oct 30 11:33:51.198790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.198417    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"42de0203-4d99-4a4a-8fff-576a0da8fd36","Type":"ContainerDied","Data":"8c11222c6b1ce3c3599424f35264fab76ba829c18bf65f64c0b43e1168d11988"}
Oct 30 11:33:51.198790 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.198439    4849 scope.go:122] "RemoveContainer" containerID="53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955"
Oct 30 11:33:51.213643 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.213624    4849 scope.go:122] "RemoveContainer" containerID="53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955"
Oct 30 11:33:51.213900 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:51.213883    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955\": container with ID starting with 53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955 not found: ID does not exist" containerID="53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955"
Oct 30 11:33:51.213954 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.213907    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955"} err="failed to get container status \"53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955\": rpc error: code = NotFound desc = could not find container \"53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955\": container with ID starting with 53350014be0d0826e3dfaad0625b9be1e1e0367bfedb79236b2654dde115c955 not found: ID does not exist"
Oct 30 11:33:51.280742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.280720    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42de0203-4d99-4a4a-8fff-576a0da8fd36-kube-api-access-k4x6t\" (UniqueName: \"kubernetes.io/projected/42de0203-4d99-4a4a-8fff-576a0da8fd36-kube-api-access-k4x6t\") pod \"42de0203-4d99-4a4a-8fff-576a0da8fd36\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") "
Oct 30 11:33:51.280842 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.280748    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42de0203-4d99-4a4a-8fff-576a0da8fd36-otc-internal\" (UniqueName: \"kubernetes.io/configmap/42de0203-4d99-4a4a-8fff-576a0da8fd36-otc-internal\") pod \"42de0203-4d99-4a4a-8fff-576a0da8fd36\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") "
Oct 30 11:33:51.280842 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.280809    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-monitoring-tls-certs\") pod \"42de0203-4d99-4a4a-8fff-576a0da8fd36\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") "
Oct 30 11:33:51.280966 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.280849    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-data-science-collector-ta-client-cert\") pod \"42de0203-4d99-4a4a-8fff-576a0da8fd36\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") "
Oct 30 11:33:51.280966 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.280928    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-tls-certs\") pod \"42de0203-4d99-4a4a-8fff-576a0da8fd36\" (UID: \"42de0203-4d99-4a4a-8fff-576a0da8fd36\") "
Oct 30 11:33:51.281247 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.281224    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42de0203-4d99-4a4a-8fff-576a0da8fd36-otc-internal" pod "42de0203-4d99-4a4a-8fff-576a0da8fd36" (UID: "42de0203-4d99-4a4a-8fff-576a0da8fd36"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:33:51.281453 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.281422    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/42de0203-4d99-4a4a-8fff-576a0da8fd36-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:51.283273 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.283239    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42de0203-4d99-4a4a-8fff-576a0da8fd36-kube-api-access-k4x6t" pod "42de0203-4d99-4a4a-8fff-576a0da8fd36" (UID: "42de0203-4d99-4a4a-8fff-576a0da8fd36"). InnerVolumeSpecName "kube-api-access-k4x6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:33:51.283466 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.283443    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-data-science-collector-ta-client-cert" pod "42de0203-4d99-4a4a-8fff-576a0da8fd36" (UID: "42de0203-4d99-4a4a-8fff-576a0da8fd36"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:51.283558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.283466    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-tls-certs" pod "42de0203-4d99-4a4a-8fff-576a0da8fd36" (UID: "42de0203-4d99-4a4a-8fff-576a0da8fd36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:51.283558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.283499    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-monitoring-tls-certs" pod "42de0203-4d99-4a4a-8fff-576a0da8fd36" (UID: "42de0203-4d99-4a4a-8fff-576a0da8fd36"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:51.382270 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.382248    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:51.382270 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.382269    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:51.382396 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.382280    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42de0203-4d99-4a4a-8fff-576a0da8fd36-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:51.382396 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.382288    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4x6t\" (UniqueName: \"kubernetes.io/projected/42de0203-4d99-4a4a-8fff-576a0da8fd36-kube-api-access-k4x6t\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:51.546141 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.546118    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:51.559812 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.559784    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:51.590808 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.590788    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:51.591177 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.591165    4849 cpu_manager.go:414] "containerMap: removing container" podUID="42de0203-4d99-4a4a-8fff-576a0da8fd36" containerName="otc-container"
Oct 30 11:33:51.591233 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.591178    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42de0203-4d99-4a4a-8fff-576a0da8fd36" containerName="otc-container"
Oct 30 11:33:51.591279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.591255    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="42de0203-4d99-4a4a-8fff-576a0da8fd36" containerName="otc-container"
Oct 30 11:33:51.595645 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.595627    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.614814 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.614310    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:33:51.614814 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.614352    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:33:51.614814 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.614448    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:33:51.614814 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.614516    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:33:51.614814 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.614573    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-dr7t6\""
Oct 30 11:33:51.616567 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.616543    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:51.684783 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.684755    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.684887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.684804    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/085468d4-bc61-415d-a275-17db1352e643-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.684887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.684878    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v277f\" (UniqueName: \"kubernetes.io/projected/085468d4-bc61-415d-a275-17db1352e643-kube-api-access-v277f\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.684977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.684946    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.685051 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.684978    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.786115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.786064    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.786115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.786097    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.786281 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.786123    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.786281 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.786174    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/085468d4-bc61-415d-a275-17db1352e643-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.786281 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.786213    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v277f\" (UniqueName: \"kubernetes.io/projected/085468d4-bc61-415d-a275-17db1352e643-kube-api-access-v277f\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.786985 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.786942    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/085468d4-bc61-415d-a275-17db1352e643-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.788827 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.788795    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.788940 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.788910    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.789279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.789257    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.810589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.810566    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v277f\" (UniqueName: \"kubernetes.io/projected/085468d4-bc61-415d-a275-17db1352e643-kube-api-access-v277f\") pod \"data-science-collector-collector-0\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:51.908230 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:51.908208    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:52.211109 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:52.211079    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:52.214743 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:33:52.214714    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085468d4_bc61_415d_a275_17db1352e643.slice/crio-406a1dd0d9f96883d79fdc54d73515ebc0b5228224783fec418e0a46c6e05dcc WatchSource:0}: Error finding container 406a1dd0d9f96883d79fdc54d73515ebc0b5228224783fec418e0a46c6e05dcc: Status 404 returned error can't find the container with id 406a1dd0d9f96883d79fdc54d73515ebc0b5228224783fec418e0a46c6e05dcc
Oct 30 11:33:52.217081 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:52.217050    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:33:52.217167 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:52.217108    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:33:52.754036 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:52.753989    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42de0203-4d99-4a4a-8fff-576a0da8fd36" path="/var/lib/kubelet/pods/42de0203-4d99-4a4a-8fff-576a0da8fd36/volumes"
Oct 30 11:33:53.211798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:53.211757    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"085468d4-bc61-415d-a275-17db1352e643","Type":"ContainerStarted","Data":"96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce"}
Oct 30 11:33:53.212316 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:53.211821    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"085468d4-bc61-415d-a275-17db1352e643","Type":"ContainerStarted","Data":"406a1dd0d9f96883d79fdc54d73515ebc0b5228224783fec418e0a46c6e05dcc"}
Oct 30 11:33:53.258318 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:53.258266    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=2.258251843 podStartE2EDuration="2.258251843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:33:53.255889444 +0000 UTC m=+2930.636338881" watchObservedRunningTime="2026-10-30 11:33:53.258251843 +0000 UTC m=+2930.638701281"
Oct 30 11:33:58.801556 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:58.801504    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:58.802114 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:58.801744    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="085468d4-bc61-415d-a275-17db1352e643" containerName="otc-container" containerID="cri-o://96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce" gracePeriod=30
Oct 30 11:33:59.051716 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.051661    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.148843 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.148814    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-data-science-collector-ta-client-cert\") pod \"085468d4-bc61-415d-a275-17db1352e643\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") "
Oct 30 11:33:59.148966 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.148863    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/085468d4-bc61-415d-a275-17db1352e643-kube-api-access-v277f\" (UniqueName: \"kubernetes.io/projected/085468d4-bc61-415d-a275-17db1352e643-kube-api-access-v277f\") pod \"085468d4-bc61-415d-a275-17db1352e643\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") "
Oct 30 11:33:59.148966 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.148951    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-tls-certs\") pod \"085468d4-bc61-415d-a275-17db1352e643\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") "
Oct 30 11:33:59.149141 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.149077    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/085468d4-bc61-415d-a275-17db1352e643-otc-internal\" (UniqueName: \"kubernetes.io/configmap/085468d4-bc61-415d-a275-17db1352e643-otc-internal\") pod \"085468d4-bc61-415d-a275-17db1352e643\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") "
Oct 30 11:33:59.149141 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.149123    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-monitoring-tls-certs\") pod \"085468d4-bc61-415d-a275-17db1352e643\" (UID: \"085468d4-bc61-415d-a275-17db1352e643\") "
Oct 30 11:33:59.149471 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.149442    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085468d4-bc61-415d-a275-17db1352e643-otc-internal" pod "085468d4-bc61-415d-a275-17db1352e643" (UID: "085468d4-bc61-415d-a275-17db1352e643"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:33:59.150991 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.150961    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-tls-certs" pod "085468d4-bc61-415d-a275-17db1352e643" (UID: "085468d4-bc61-415d-a275-17db1352e643"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:59.150991 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.150971    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/085468d4-bc61-415d-a275-17db1352e643-kube-api-access-v277f" pod "085468d4-bc61-415d-a275-17db1352e643" (UID: "085468d4-bc61-415d-a275-17db1352e643"). InnerVolumeSpecName "kube-api-access-v277f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:33:59.151190 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.150992    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-data-science-collector-ta-client-cert" pod "085468d4-bc61-415d-a275-17db1352e643" (UID: "085468d4-bc61-415d-a275-17db1352e643"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:59.151190 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.151107    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-monitoring-tls-certs" pod "085468d4-bc61-415d-a275-17db1352e643" (UID: "085468d4-bc61-415d-a275-17db1352e643"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:33:59.244712 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.244688    4849 generic.go:356] "Generic (PLEG): container finished" podID="085468d4-bc61-415d-a275-17db1352e643" containerID="96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce" exitCode=0
Oct 30 11:33:59.244810 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.244735    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.244810 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.244773    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"085468d4-bc61-415d-a275-17db1352e643","Type":"ContainerDied","Data":"96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce"}
Oct 30 11:33:59.244879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.244822    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"085468d4-bc61-415d-a275-17db1352e643","Type":"ContainerDied","Data":"406a1dd0d9f96883d79fdc54d73515ebc0b5228224783fec418e0a46c6e05dcc"}
Oct 30 11:33:59.244879 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.244845    4849 scope.go:122] "RemoveContainer" containerID="96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce"
Oct 30 11:33:59.249940 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.249918    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:59.249940 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.249938    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v277f\" (UniqueName: \"kubernetes.io/projected/085468d4-bc61-415d-a275-17db1352e643-kube-api-access-v277f\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:59.250140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.249950    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:59.250140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.249962    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/085468d4-bc61-415d-a275-17db1352e643-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:59.250140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.249972    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/085468d4-bc61-415d-a275-17db1352e643-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:33:59.253686 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.253671    4849 scope.go:122] "RemoveContainer" containerID="96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce"
Oct 30 11:33:59.253933 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:33:59.253911    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce\": container with ID starting with 96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce not found: ID does not exist" containerID="96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce"
Oct 30 11:33:59.254013 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.253938    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce"} err="failed to get container status \"96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce\": rpc error: code = NotFound desc = could not find container \"96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce\": container with ID starting with 96b46001b1a19058a18ccf9f91d92002063b5357118241d9044ffc15bdd499ce not found: ID does not exist"
Oct 30 11:33:59.299317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.299292    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:59.306012 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.305961    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:59.342301 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.342277    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:59.342620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.342607    4849 cpu_manager.go:414] "containerMap: removing container" podUID="085468d4-bc61-415d-a275-17db1352e643" containerName="otc-container"
Oct 30 11:33:59.342663 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.342621    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="085468d4-bc61-415d-a275-17db1352e643" containerName="otc-container"
Oct 30 11:33:59.342712 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.342703    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="085468d4-bc61-415d-a275-17db1352e643" containerName="otc-container"
Oct 30 11:33:59.346870 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.346857    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.365373 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.365350    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:33:59.365485 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.365371    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:33:59.365606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.365589    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-dr7t6\""
Oct 30 11:33:59.365679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.365646    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:33:59.365808 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.365793    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:33:59.367859 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.367839    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:59.451969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.451938    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/accca4e4-70fd-4e47-ab6f-1b82b60397eb-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.452108 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.452013    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.452191 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.452103    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.452191 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.452139    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvc6c\" (UniqueName: \"kubernetes.io/projected/accca4e4-70fd-4e47-ab6f-1b82b60397eb-kube-api-access-cvc6c\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.452303 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.452222    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.553386 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.553356    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.553504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.553391    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvc6c\" (UniqueName: \"kubernetes.io/projected/accca4e4-70fd-4e47-ab6f-1b82b60397eb-kube-api-access-cvc6c\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.553504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.553418    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.553504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.553490    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/accca4e4-70fd-4e47-ab6f-1b82b60397eb-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.553638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.553533    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.554427 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.554402    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/accca4e4-70fd-4e47-ab6f-1b82b60397eb-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.556072 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.556000    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.556072 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.556045    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.556072 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.556053    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.577685 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.577664    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvc6c\" (UniqueName: \"kubernetes.io/projected/accca4e4-70fd-4e47-ab6f-1b82b60397eb-kube-api-access-cvc6c\") pod \"data-science-collector-collector-0\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.658807 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.658786    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:33:59.959968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.959944    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:33:59.961990 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:33:59.961964    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaccca4e4_70fd_4e47_ab6f_1b82b60397eb.slice/crio-016cf67227f3a6f756f59cd346d767d63fa242c7c606a0ac9b1ae8cb4a069c2e WatchSource:0}: Error finding container 016cf67227f3a6f756f59cd346d767d63fa242c7c606a0ac9b1ae8cb4a069c2e: Status 404 returned error can't find the container with id 016cf67227f3a6f756f59cd346d767d63fa242c7c606a0ac9b1ae8cb4a069c2e
Oct 30 11:33:59.964700 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.964667    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:33:59.964810 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:33:59.964725    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:34:00.251805 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:00.251725    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"accca4e4-70fd-4e47-ab6f-1b82b60397eb","Type":"ContainerStarted","Data":"a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c"}
Oct 30 11:34:00.251805 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:00.251779    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"accca4e4-70fd-4e47-ab6f-1b82b60397eb","Type":"ContainerStarted","Data":"016cf67227f3a6f756f59cd346d767d63fa242c7c606a0ac9b1ae8cb4a069c2e"}
Oct 30 11:34:00.297568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:00.297517    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.297502832 podStartE2EDuration="1.297502832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:34:00.295958384 +0000 UTC m=+2937.676407824" watchObservedRunningTime="2026-10-30 11:34:00.297502832 +0000 UTC m=+2937.677952335"
Oct 30 11:34:00.753533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:00.753502    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="085468d4-bc61-415d-a275-17db1352e643" path="/var/lib/kubelet/pods/085468d4-bc61-415d-a275-17db1352e643/volumes"
Oct 30 11:34:01.549882 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:01.549849    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:01.550280 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:01.550150    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="alertmanager" containerID="cri-o://1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222" gracePeriod=120
Oct 30 11:34:01.550331 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:01.550303    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="config-reloader" containerID="cri-o://ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc" gracePeriod=120
Oct 30 11:34:01.566440 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:01.565417    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:34:01.566440 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:01.565739    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="prometheus" containerID="cri-o://3925665e4f1d7bb6143259e64973cd40053a63d043e71e5dbbbc20b6c1965499" gracePeriod=600
Oct 30 11:34:01.566440 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:01.565931    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="thanos-sidecar" containerID="cri-o://cff2844bdfa862e991dfdeaba342920020fd80708cafdf670eb5aa89f0ea1b61" gracePeriod=600
Oct 30 11:34:01.566440 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:01.566015    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="config-reloader" containerID="cri-o://de9db7a8b51450df818f5623dea34d77dd08e316a869dd2b4c0cc4801fe8b8ec" gracePeriod=600
Oct 30 11:34:02.190717 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.190690    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:02.275460 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.275429    4849 generic.go:356] "Generic (PLEG): container finished" podID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerID="cff2844bdfa862e991dfdeaba342920020fd80708cafdf670eb5aa89f0ea1b61" exitCode=0
Oct 30 11:34:02.275460 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.275458    4849 generic.go:356] "Generic (PLEG): container finished" podID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerID="de9db7a8b51450df818f5623dea34d77dd08e316a869dd2b4c0cc4801fe8b8ec" exitCode=0
Oct 30 11:34:02.275460 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.275467    4849 generic.go:356] "Generic (PLEG): container finished" podID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerID="3925665e4f1d7bb6143259e64973cd40053a63d043e71e5dbbbc20b6c1965499" exitCode=0
Oct 30 11:34:02.275760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.275504    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerDied","Data":"cff2844bdfa862e991dfdeaba342920020fd80708cafdf670eb5aa89f0ea1b61"}
Oct 30 11:34:02.275760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.275568    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerDied","Data":"de9db7a8b51450df818f5623dea34d77dd08e316a869dd2b4c0cc4801fe8b8ec"}
Oct 30 11:34:02.275760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.275600    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerDied","Data":"3925665e4f1d7bb6143259e64973cd40053a63d043e71e5dbbbc20b6c1965499"}
Oct 30 11:34:02.277926 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.277904    4849 generic.go:356] "Generic (PLEG): container finished" podID="656ad963-6770-4df3-bd17-005b40732b69" containerID="ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc" exitCode=0
Oct 30 11:34:02.278071 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.278012    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerDied","Data":"ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc"}
Oct 30 11:34:02.278145 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.278077    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="accca4e4-70fd-4e47-ab6f-1b82b60397eb" containerName="otc-container" containerID="cri-o://a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c" gracePeriod=30
Oct 30 11:34:02.393061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.393039    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:34:02.479477 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479442    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479493    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479561    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-tls-assets\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-tls-assets\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479605    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-kube-api-access-2gqhp\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-kube-api-access-2gqhp\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479633    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1026b4d2-2d29-4321-9e85-5489db9ac13a-config-out\" (UniqueName: \"kubernetes.io/empty-dir/1026b4d2-2d29-4321-9e85-5489db9ac13a-config-out\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479673    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479719    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-config\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479748    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479812    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479845    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-secret-prometheus-web-tls-ca\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479890    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-thanos-prometheus-http-client-file\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.479967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.479923    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.480300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.480049    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") pod \"1026b4d2-2d29-4321-9e85-5489db9ac13a\" (UID: \"1026b4d2-2d29-4321-9e85-5489db9ac13a\") "
Oct 30 11:34:02.480300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.480167    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-2" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:02.481725 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.480362    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.481725 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.481375    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-0" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:02.481725 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.481510    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-1" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:02.482562 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.482519    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.483964 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.483916    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-kube-api-access-2gqhp" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "kube-api-access-2gqhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:02.484477 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.484447    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.484882 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.484844    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-thanos-prometheus-http-client-file" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.485564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.485537    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-secret-prometheus-web-tls-ca" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.485797 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.485767    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-tls-assets" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:02.486190 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.486161    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-config" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.486297 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.486202    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1026b4d2-2d29-4321-9e85-5489db9ac13a-config-out" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:34:02.489658 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.489637    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "pvc-c799cc48-125a-421c-8fc6-8a4d017a7653". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:34:02.505759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.505732    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config" pod "1026b4d2-2d29-4321-9e85-5489db9ac13a" (UID: "1026b4d2-2d29-4321-9e85-5489db9ac13a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.517452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.517425    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:02.581425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581396    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581429    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581455    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581470    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581484    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581500    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1026b4d2-2d29-4321-9e85-5489db9ac13a-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581532    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581547    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581567    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/1026b4d2-2d29-4321-9e85-5489db9ac13a-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581583    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581596    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gqhp\" (UniqueName: \"kubernetes.io/projected/1026b4d2-2d29-4321-9e85-5489db9ac13a-kube-api-access-2gqhp\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.581742 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.581609    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1026b4d2-2d29-4321-9e85-5489db9ac13a-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.601193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.601174    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-c799cc48-125a-421c-8fc6-8a4d017a7653" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:34:02.682581 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.682555    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-monitoring-tls-certs\") pod \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") "
Oct 30 11:34:02.682691 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.682628    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/accca4e4-70fd-4e47-ab6f-1b82b60397eb-otc-internal\" (UniqueName: \"kubernetes.io/configmap/accca4e4-70fd-4e47-ab6f-1b82b60397eb-otc-internal\") pod \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") "
Oct 30 11:34:02.682691 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.682681    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/accca4e4-70fd-4e47-ab6f-1b82b60397eb-kube-api-access-cvc6c\" (UniqueName: \"kubernetes.io/projected/accca4e4-70fd-4e47-ab6f-1b82b60397eb-kube-api-access-cvc6c\") pod \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") "
Oct 30 11:34:02.682791 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.682775    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-data-science-collector-ta-client-cert\") pod \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") "
Oct 30 11:34:02.682866 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.682844    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-tls-certs\") pod \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\" (UID: \"accca4e4-70fd-4e47-ab6f-1b82b60397eb\") "
Oct 30 11:34:02.682994 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.682970    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/accca4e4-70fd-4e47-ab6f-1b82b60397eb-otc-internal" pod "accca4e4-70fd-4e47-ab6f-1b82b60397eb" (UID: "accca4e4-70fd-4e47-ab6f-1b82b60397eb"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:02.683152 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.683133    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-c799cc48-125a-421c-8fc6-8a4d017a7653\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0da8893e7ad2dfa48\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-bc5b0eea9a26be0763bd2234369df1d0cb0bca5610c96b98228e00775a514ae6\""
Oct 30 11:34:02.683252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.683158    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/accca4e4-70fd-4e47-ab6f-1b82b60397eb-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.684601 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.684578    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-monitoring-tls-certs" pod "accca4e4-70fd-4e47-ab6f-1b82b60397eb" (UID: "accca4e4-70fd-4e47-ab6f-1b82b60397eb"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.684852 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.684826    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-data-science-collector-ta-client-cert" pod "accca4e4-70fd-4e47-ab6f-1b82b60397eb" (UID: "accca4e4-70fd-4e47-ab6f-1b82b60397eb"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.684915 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.684860    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-tls-certs" pod "accca4e4-70fd-4e47-ab6f-1b82b60397eb" (UID: "accca4e4-70fd-4e47-ab6f-1b82b60397eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:02.684953 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.684921    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/accca4e4-70fd-4e47-ab6f-1b82b60397eb-kube-api-access-cvc6c" pod "accca4e4-70fd-4e47-ab6f-1b82b60397eb" (UID: "accca4e4-70fd-4e47-ab6f-1b82b60397eb"). InnerVolumeSpecName "kube-api-access-cvc6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:02.783739 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.783719    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.783833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.783742    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvc6c\" (UniqueName: \"kubernetes.io/projected/accca4e4-70fd-4e47-ab6f-1b82b60397eb-kube-api-access-cvc6c\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.783833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.783756    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:02.783833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:02.783771    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/accca4e4-70fd-4e47-ab6f-1b82b60397eb-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:03.123818 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.123781    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.116:9093/-/ready\": dial tcp 10.132.0.116:9093: connect: connection refused"
Oct 30 11:34:03.285728 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.285693    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1" event={"ID":"1026b4d2-2d29-4321-9e85-5489db9ac13a","Type":"ContainerDied","Data":"5143d51bfcc27a6b905b93c7f4a970bd90239857364a9f1b208f13e007219608"}
Oct 30 11:34:03.285922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.285783    4849 scope.go:122] "RemoveContainer" containerID="cff2844bdfa862e991dfdeaba342920020fd80708cafdf670eb5aa89f0ea1b61"
Oct 30 11:34:03.285922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.285703    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"
Oct 30 11:34:03.287396 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.287367    4849 generic.go:356] "Generic (PLEG): container finished" podID="accca4e4-70fd-4e47-ab6f-1b82b60397eb" containerID="a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c" exitCode=0
Oct 30 11:34:03.287502 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.287419    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:03.287502 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.287450    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"accca4e4-70fd-4e47-ab6f-1b82b60397eb","Type":"ContainerDied","Data":"a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c"}
Oct 30 11:34:03.287502 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.287488    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"accca4e4-70fd-4e47-ab6f-1b82b60397eb","Type":"ContainerDied","Data":"016cf67227f3a6f756f59cd346d767d63fa242c7c606a0ac9b1ae8cb4a069c2e"}
Oct 30 11:34:03.294508 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.294491    4849 scope.go:122] "RemoveContainer" containerID="de9db7a8b51450df818f5623dea34d77dd08e316a869dd2b4c0cc4801fe8b8ec"
Oct 30 11:34:03.302400 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.302381    4849 scope.go:122] "RemoveContainer" containerID="3925665e4f1d7bb6143259e64973cd40053a63d043e71e5dbbbc20b6c1965499"
Oct 30 11:34:03.310403 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.310386    4849 scope.go:122] "RemoveContainer" containerID="9f122db444f1bc4380944fed9c6c0d3b529a9181d6d90985ad9857ce073146b1"
Oct 30 11:34:03.318084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.318067    4849 scope.go:122] "RemoveContainer" containerID="a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c"
Oct 30 11:34:03.325321 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.325301    4849 scope.go:122] "RemoveContainer" containerID="a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c"
Oct 30 11:34:03.325589 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:03.325570    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c\": container with ID starting with a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c not found: ID does not exist" containerID="a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c"
Oct 30 11:34:03.325637 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.325595    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c"} err="failed to get container status \"a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c\": rpc error: code = NotFound desc = could not find container \"a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c\": container with ID starting with a5ee9c4dbd5481beecfa9b707c07be39cf4a805beecc019136fbe8d85866a24c not found: ID does not exist"
Oct 30 11:34:03.343452 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.343427    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:34:03.357076 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.357053    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-1"]
Oct 30 11:34:03.384857 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.384807    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:03.390616 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:03.390595    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:04.753736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:04.753696    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" path="/var/lib/kubelet/pods/1026b4d2-2d29-4321-9e85-5489db9ac13a/volumes"
Oct 30 11:34:04.754298 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:04.754279    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="accca4e4-70fd-4e47-ab6f-1b82b60397eb" path="/var/lib/kubelet/pods/accca4e4-70fd-4e47-ab6f-1b82b60397eb/volumes"
Oct 30 11:34:08.110314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110285    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110791    4849 cpu_manager.go:414] "containerMap: removing container" podUID="accca4e4-70fd-4e47-ab6f-1b82b60397eb" containerName="otc-container"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110811    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="accca4e4-70fd-4e47-ab6f-1b82b60397eb" containerName="otc-container"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110845    4849 cpu_manager.go:414] "containerMap: removing container" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="init-config-reloader"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110898    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="init-config-reloader"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110911    4849 cpu_manager.go:414] "containerMap: removing container" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="config-reloader"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110916    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="config-reloader"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110932    4849 cpu_manager.go:414] "containerMap: removing container" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="thanos-sidecar"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110937    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="thanos-sidecar"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110951    4849 cpu_manager.go:414] "containerMap: removing container" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="prometheus"
Oct 30 11:34:08.111063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.110958    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="prometheus"
Oct 30 11:34:08.111492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.111085    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="thanos-sidecar"
Oct 30 11:34:08.111492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.111099    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="prometheus"
Oct 30 11:34:08.111492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.111112    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="accca4e4-70fd-4e47-ab6f-1b82b60397eb" containerName="otc-container"
Oct 30 11:34:08.111492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.111124    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="1026b4d2-2d29-4321-9e85-5489db9ac13a" containerName="config-reloader"
Oct 30 11:34:08.116232 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.116209    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.122797 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.122770    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.116:9093/-/ready\": dial tcp 10.132.0.116:9093: connect: connection refused"
Oct 30 11:34:08.137081 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.137061    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:34:08.150242 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.150221    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:34:08.152246 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.152223    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:08.163559 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.163539    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:34:08.163559 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.163540    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:34:08.163831 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.163815    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-phltc\""
Oct 30 11:34:08.226558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.226530    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.226685 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.226568    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5e5b71e0-7616-4564-bdeb-f2436f2b9193-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.226685 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.226598    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.226803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.226683    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.226803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.226713    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsvmm\" (UniqueName: \"kubernetes.io/projected/5e5b71e0-7616-4564-bdeb-f2436f2b9193-kube-api-access-wsvmm\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.327674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.327642    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.327674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.327677    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsvmm\" (UniqueName: \"kubernetes.io/projected/5e5b71e0-7616-4564-bdeb-f2436f2b9193-kube-api-access-wsvmm\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.327887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.327738    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.327887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.327760    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5e5b71e0-7616-4564-bdeb-f2436f2b9193-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.327887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.327795    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.328649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.328615    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5e5b71e0-7616-4564-bdeb-f2436f2b9193-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.330300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.330277    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.330300 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.330286    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.330488 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.330376    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.352196 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.352173    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsvmm\" (UniqueName: \"kubernetes.io/projected/5e5b71e0-7616-4564-bdeb-f2436f2b9193-kube-api-access-wsvmm\") pod \"data-science-collector-collector-0\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.430679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.430621    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:08.736676 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.736626    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:08.741521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.741491    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:34:08.741611 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:08.741546    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:34:09.322575 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:09.322539    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5e5b71e0-7616-4564-bdeb-f2436f2b9193","Type":"ContainerStarted","Data":"9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65"}
Oct 30 11:34:09.322970 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:09.322597    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5e5b71e0-7616-4564-bdeb-f2436f2b9193","Type":"ContainerStarted","Data":"dbe6d283f213fcef18c70d00c5af17f6f5d4dae14972fb18c7ceb2cdbe551a34"}
Oct 30 11:34:09.374760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:09.374708    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.3746945099999999 podStartE2EDuration="1.37469451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:34:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:34:09.373162771 +0000 UTC m=+2946.753612206" watchObservedRunningTime="2026-10-30 11:34:09.37469451 +0000 UTC m=+2946.755143949"
Oct 30 11:34:09.432093 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:09.432058    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:34:11.512097 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.512059    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:34:11.516060 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.516039    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.532777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.532752    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 11:34:11.536381 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.536359    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 11:34:11.536381 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.536400    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 11:34:11.537167 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.537149    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:34:11.548207 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.548177    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 11:34:11.548207 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.548182    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 11:34:11.548421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.548230    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:34:11.548421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.548274    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-jf5dq\""
Oct 30 11:34:11.548421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.548298    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 11:34:11.633243 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.633222    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 11:34:11.633332 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.633273    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 11:34:11.653639 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.653618    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.653762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.653651    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.653762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.653675    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.653762 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.653752    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdzg6\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-kube-api-access-hdzg6\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.653928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.653864    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.653928 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.653898    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.654075 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.653965    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.654075 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.654050    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.654184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.654094    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.654184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.654130    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.654362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.654223    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.654362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.654276    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.686897 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:11.686864    4849 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod656ad963_6770_4df3_bd17_005b40732b69.slice/crio-1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 11:34:11.755336 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755310    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdzg6\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-kube-api-access-hdzg6\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755455 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755347    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755455 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755365    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755455 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755381    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755414    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755603    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755725 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755655    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755725 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755705    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755832 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755782    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755846    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.755922 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755896    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.756048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.755926    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.756130 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.756105    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.756845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.756820    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.757237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.757209    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.758672 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.758644    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.758773 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.758698    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.758858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.758790    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.759297 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.759228    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.759297 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.759229    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.759297 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.759283    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.760582 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.760545    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.761054 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.761015    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.780144 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.780081    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdzg6\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-kube-api-access-hdzg6\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:11.820005 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.819980    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:11.957983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.957955    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-config-out\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-config-out\") pod \"656ad963-6770-4df3-bd17-005b40732b69\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") "
Oct 30 11:34:11.958137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.958012    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-tls-assets\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-tls-assets\") pod \"656ad963-6770-4df3-bd17-005b40732b69\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") "
Oct 30 11:34:11.958137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.958075    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-config-volume\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-config-volume\") pod \"656ad963-6770-4df3-bd17-005b40732b69\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") "
Oct 30 11:34:11.958137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.958117    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-kube-api-access-tq8wr\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-kube-api-access-tq8wr\") pod \"656ad963-6770-4df3-bd17-005b40732b69\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") "
Oct 30 11:34:11.958309 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.958229    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-web-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-web-config\") pod \"656ad963-6770-4df3-bd17-005b40732b69\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") "
Oct 30 11:34:11.958365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.958315    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-cluster-tls-config\") pod \"656ad963-6770-4df3-bd17-005b40732b69\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") "
Oct 30 11:34:11.958421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.958359    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-alertmanager-data-science-monitoringstack-db\") pod \"656ad963-6770-4df3-bd17-005b40732b69\" (UID: \"656ad963-6770-4df3-bd17-005b40732b69\") "
Oct 30 11:34:11.958929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.958904    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-alertmanager-data-science-monitoringstack-db" pod "656ad963-6770-4df3-bd17-005b40732b69" (UID: "656ad963-6770-4df3-bd17-005b40732b69"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:34:11.960681 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.960650    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-tls-assets" pod "656ad963-6770-4df3-bd17-005b40732b69" (UID: "656ad963-6770-4df3-bd17-005b40732b69"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:11.960798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.960655    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-kube-api-access-tq8wr" pod "656ad963-6770-4df3-bd17-005b40732b69" (UID: "656ad963-6770-4df3-bd17-005b40732b69"). InnerVolumeSpecName "kube-api-access-tq8wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:11.960798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.960691    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-config-out" pod "656ad963-6770-4df3-bd17-005b40732b69" (UID: "656ad963-6770-4df3-bd17-005b40732b69"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:34:11.960798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.960729    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-config-volume" pod "656ad963-6770-4df3-bd17-005b40732b69" (UID: "656ad963-6770-4df3-bd17-005b40732b69"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:11.964826 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.964804    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-cluster-tls-config" pod "656ad963-6770-4df3-bd17-005b40732b69" (UID: "656ad963-6770-4df3-bd17-005b40732b69"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:11.971605 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:11.971585    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-web-config" pod "656ad963-6770-4df3-bd17-005b40732b69" (UID: "656ad963-6770-4df3-bd17-005b40732b69"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:12.059126 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.059103    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:12.059126 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.059123    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq8wr\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-kube-api-access-tq8wr\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:12.059251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.059133    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:12.059251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.059141    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/656ad963-6770-4df3-bd17-005b40732b69-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:12.059251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.059150    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:12.059251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.059159    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/656ad963-6770-4df3-bd17-005b40732b69-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:12.059251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.059169    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/656ad963-6770-4df3-bd17-005b40732b69-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:12.340322 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.340257    4849 generic.go:356] "Generic (PLEG): container finished" podID="656ad963-6770-4df3-bd17-005b40732b69" containerID="1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222" exitCode=0
Oct 30 11:34:12.340413 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.340341    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:12.340413 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.340346    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerDied","Data":"1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222"}
Oct 30 11:34:12.340488 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.340443    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"656ad963-6770-4df3-bd17-005b40732b69","Type":"ContainerDied","Data":"7f42b2bc0e870f5974f7b0f1cbca2f38ecc47e20d6722750261f6cbbd7d3085a"}
Oct 30 11:34:12.340488 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.340474    4849 scope.go:122] "RemoveContainer" containerID="ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc"
Oct 30 11:34:12.351370 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.351349    4849 scope.go:122] "RemoveContainer" containerID="1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222"
Oct 30 11:34:12.359526 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.359500    4849 scope.go:122] "RemoveContainer" containerID="e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509"
Oct 30 11:34:12.367186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.367167    4849 scope.go:122] "RemoveContainer" containerID="ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc"
Oct 30 11:34:12.367470 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:12.367449    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc\": container with ID starting with ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc not found: ID does not exist" containerID="ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc"
Oct 30 11:34:12.367532 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.367478    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc"} err="failed to get container status \"ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc\": rpc error: code = NotFound desc = could not find container \"ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc\": container with ID starting with ffb1ef8bf69f4e359745be8673bdba25659a7359da7d81d075997695381e92bc not found: ID does not exist"
Oct 30 11:34:12.367532 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.367494    4849 scope.go:122] "RemoveContainer" containerID="1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222"
Oct 30 11:34:12.367744 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:12.367724    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222\": container with ID starting with 1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222 not found: ID does not exist" containerID="1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222"
Oct 30 11:34:12.367827 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.367755    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222"} err="failed to get container status \"1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222\": rpc error: code = NotFound desc = could not find container \"1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222\": container with ID starting with 1e19a6e410f4cad62bb3ba9d63455cbdff5a97c06c594a7a1da5cb59c467f222 not found: ID does not exist"
Oct 30 11:34:12.367827 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.367777    4849 scope.go:122] "RemoveContainer" containerID="e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509"
Oct 30 11:34:12.368121 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:12.368013    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509\": container with ID starting with e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509 not found: ID does not exist" containerID="e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509"
Oct 30 11:34:12.368173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.368126    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509"} err="failed to get container status \"e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509\": rpc error: code = NotFound desc = could not find container \"e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509\": container with ID starting with e44c9c964a578d3eea77579b2408b086f916c47ac3e54f534c082c440a3da509 not found: ID does not exist"
Oct 30 11:34:12.406067 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.406044    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:12.419654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.419632    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:12.753795 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:12.753723    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="656ad963-6770-4df3-bd17-005b40732b69" path="/var/lib/kubelet/pods/656ad963-6770-4df3-bd17-005b40732b69/volumes"
Oct 30 11:34:13.560478 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.560442    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:13.560845 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.560829    4849 cpu_manager.go:414] "containerMap: removing container" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="alertmanager"
Oct 30 11:34:13.560927 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.560847    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="alertmanager"
Oct 30 11:34:13.560927 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.560869    4849 cpu_manager.go:414] "containerMap: removing container" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="init-config-reloader"
Oct 30 11:34:13.560927 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.560877    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="init-config-reloader"
Oct 30 11:34:13.560927 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.560905    4849 cpu_manager.go:414] "containerMap: removing container" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="config-reloader"
Oct 30 11:34:13.560927 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.560913    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="config-reloader"
Oct 30 11:34:13.561218 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.561007    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="alertmanager"
Oct 30 11:34:13.561218 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.561042    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="656ad963-6770-4df3-bd17-005b40732b69" containerName="config-reloader"
Oct 30 11:34:13.565886 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.565867    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.582688 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.582669    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:34:13.582822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.582776    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 11:34:13.582822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.582801    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 11:34:13.582822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.582808    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 11:34:13.582984 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.582834    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-ltmbg\""
Oct 30 11:34:13.599650 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.599629    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:13.668871 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.668850    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.668989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.668876    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.668989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.668901    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.668989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.668972    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.669136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.669052    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.669136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.669071    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.669136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.669099    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzzbk\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-kube-api-access-zzzbk\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.769740 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.769714    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.770099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.769743    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.770099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.769762    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.770099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.769790    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzzbk\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-kube-api-access-zzzbk\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.770099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.769904    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.770099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.769932    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.770099 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.769970    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.770414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.770232    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.772173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.772151    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.772333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.772301    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.772468 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.772442    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.772608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.772590    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.772661 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.772650    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.793692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.793672    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzzbk\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-kube-api-access-zzzbk\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:13.882826 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:13.882774    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:14.215191 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:14.215164    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:14.216950 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:34:14.216923    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bc7634_538b_44d8_8620_edb1fa360d7e.slice/crio-9aefe06267d4dcac7f217eaf16cd12f6363dbd1f7f68e611596c3a0ba32de9bb WatchSource:0}: Error finding container 9aefe06267d4dcac7f217eaf16cd12f6363dbd1f7f68e611596c3a0ba32de9bb: Status 404 returned error can't find the container with id 9aefe06267d4dcac7f217eaf16cd12f6363dbd1f7f68e611596c3a0ba32de9bb
Oct 30 11:34:14.352534 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:14.352505    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerStarted","Data":"9aefe06267d4dcac7f217eaf16cd12f6363dbd1f7f68e611596c3a0ba32de9bb"}
Oct 30 11:34:15.281313 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.281278    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:15.313188 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.313158    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:15.359491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.359458    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerStarted","Data":"bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3"}
Oct 30 11:34:15.381712 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.381689    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:15.381833 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.381815    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:15.397144 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.397124    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:15.503795 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.503772    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5f4e377c3e259c071aab429e31ca61846b879cc3476e2f02b70e888ccdcdb590/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:15.535036 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.534971    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:15.738309 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:15.738283    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:16.082798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:16.082768    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:34:16.084212 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:34:16.084181    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65e9a68_3e4f_42b8_986e_ac3163a2ecdf.slice/crio-5d0cd899c13126734847b60e139ded82a26c232d543d5e768fe653900c6b46ab WatchSource:0}: Error finding container 5d0cd899c13126734847b60e139ded82a26c232d543d5e768fe653900c6b46ab: Status 404 returned error can't find the container with id 5d0cd899c13126734847b60e139ded82a26c232d543d5e768fe653900c6b46ab
Oct 30 11:34:16.365880 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:16.365826    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerStarted","Data":"5d0cd899c13126734847b60e139ded82a26c232d543d5e768fe653900c6b46ab"}
Oct 30 11:34:17.371690 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:17.371656    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerStarted","Data":"7bc379f2f495a05f57de3b140c3a24456a2ff0cbf488567e50154347198d4e72"}
Oct 30 11:34:18.378312 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:18.378280    4849 generic.go:356] "Generic (PLEG): container finished" podID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerID="bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3" exitCode=0
Oct 30 11:34:18.378625 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:18.378360    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerDied","Data":"bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3"}
Oct 30 11:34:19.385853 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:19.385818    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerStarted","Data":"4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565"}
Oct 30 11:34:19.386249 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:19.385877    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerStarted","Data":"bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64"}
Oct 30 11:34:19.386249 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:19.386062    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:19.463827 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:19.463778    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podStartSLOduration=2.6783609200000003 podStartE2EDuration="6.46376524s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.78540432s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:34:19.449256173 +0000 UTC m=+2956.829705612" watchObservedRunningTime="2026-10-30 11:34:19.46376524 +0000 UTC m=+2956.844214679"
Oct 30 11:34:20.394154 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:20.394120    4849 generic.go:356] "Generic (PLEG): container finished" podID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerID="7bc379f2f495a05f57de3b140c3a24456a2ff0cbf488567e50154347198d4e72" exitCode=0
Oct 30 11:34:20.394503 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:20.394200    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerDied","Data":"7bc379f2f495a05f57de3b140c3a24456a2ff0cbf488567e50154347198d4e72"}
Oct 30 11:34:21.401269 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:21.401235    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerStarted","Data":"ac48755d426db1f7f224406cd0128d6b8fcb70057b969875c8ebb7acb2c16211"}
Oct 30 11:34:21.401766 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:21.401325    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerStarted","Data":"cc3f8e4cdee469e8e376575a980757458e156380f0865560d6f1f43b0eb9cda9"}
Oct 30 11:34:21.401766 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:21.401349    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerStarted","Data":"01e16fd02a9d3708367f021dd22a1c59372672952cb6f46135775d34cf4f27e8"}
Oct 30 11:34:21.482898 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:21.482849    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podStartSLOduration=7.098162241 podStartE2EDuration="10.48283736s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.384675119s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:34:21.468260849 +0000 UTC m=+2958.848710313" watchObservedRunningTime="2026-10-30 11:34:21.48283736 +0000 UTC m=+2958.863286799"
Oct 30 11:34:25.395844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:25.395804    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:25.738734 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:25.738654    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:29.012931 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.012901    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:29.013446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.013228    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="5e5b71e0-7616-4564-bdeb-f2436f2b9193" containerName="otc-container" containerID="cri-o://9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65" gracePeriod=30
Oct 30 11:34:29.268187 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.268137    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.293439 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.293409    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-tls-certs\") pod \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") "
Oct 30 11:34:29.293554 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.293513    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e5b71e0-7616-4564-bdeb-f2436f2b9193-otc-internal\" (UniqueName: \"kubernetes.io/configmap/5e5b71e0-7616-4564-bdeb-f2436f2b9193-otc-internal\") pod \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") "
Oct 30 11:34:29.293595 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.293558    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-data-science-collector-ta-client-cert\") pod \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") "
Oct 30 11:34:29.293595 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.293579    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e5b71e0-7616-4564-bdeb-f2436f2b9193-kube-api-access-wsvmm\" (UniqueName: \"kubernetes.io/projected/5e5b71e0-7616-4564-bdeb-f2436f2b9193-kube-api-access-wsvmm\") pod \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") "
Oct 30 11:34:29.293661 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.293610    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-monitoring-tls-certs\") pod \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\" (UID: \"5e5b71e0-7616-4564-bdeb-f2436f2b9193\") "
Oct 30 11:34:29.293916 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.293892    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e5b71e0-7616-4564-bdeb-f2436f2b9193-otc-internal" pod "5e5b71e0-7616-4564-bdeb-f2436f2b9193" (UID: "5e5b71e0-7616-4564-bdeb-f2436f2b9193"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:29.295553 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.295526    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-tls-certs" pod "5e5b71e0-7616-4564-bdeb-f2436f2b9193" (UID: "5e5b71e0-7616-4564-bdeb-f2436f2b9193"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:29.295684 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.295649    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-monitoring-tls-certs" pod "5e5b71e0-7616-4564-bdeb-f2436f2b9193" (UID: "5e5b71e0-7616-4564-bdeb-f2436f2b9193"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:29.295778 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.295718    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-data-science-collector-ta-client-cert" pod "5e5b71e0-7616-4564-bdeb-f2436f2b9193" (UID: "5e5b71e0-7616-4564-bdeb-f2436f2b9193"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:29.295778 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.295737    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e5b71e0-7616-4564-bdeb-f2436f2b9193-kube-api-access-wsvmm" pod "5e5b71e0-7616-4564-bdeb-f2436f2b9193" (UID: "5e5b71e0-7616-4564-bdeb-f2436f2b9193"). InnerVolumeSpecName "kube-api-access-wsvmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:29.395064 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.395015    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:29.395064 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.395064    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:29.395273 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.395078    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/5e5b71e0-7616-4564-bdeb-f2436f2b9193-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:29.395273 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.395094    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/5e5b71e0-7616-4564-bdeb-f2436f2b9193-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:29.395273 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.395108    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsvmm\" (UniqueName: \"kubernetes.io/projected/5e5b71e0-7616-4564-bdeb-f2436f2b9193-kube-api-access-wsvmm\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:29.447265 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.447232    4849 generic.go:356] "Generic (PLEG): container finished" podID="5e5b71e0-7616-4564-bdeb-f2436f2b9193" containerID="9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65" exitCode=0
Oct 30 11:34:29.447391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.447278    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.447391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.447316    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5e5b71e0-7616-4564-bdeb-f2436f2b9193","Type":"ContainerDied","Data":"9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65"}
Oct 30 11:34:29.447391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.447368    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"5e5b71e0-7616-4564-bdeb-f2436f2b9193","Type":"ContainerDied","Data":"dbe6d283f213fcef18c70d00c5af17f6f5d4dae14972fb18c7ceb2cdbe551a34"}
Oct 30 11:34:29.447391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.447390    4849 scope.go:122] "RemoveContainer" containerID="9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65"
Oct 30 11:34:29.457490 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.457476    4849 scope.go:122] "RemoveContainer" containerID="9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65"
Oct 30 11:34:29.457765 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:29.457748    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65\": container with ID starting with 9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65 not found: ID does not exist" containerID="9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65"
Oct 30 11:34:29.457820 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.457776    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65"} err="failed to get container status \"9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65\": rpc error: code = NotFound desc = could not find container \"9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65\": container with ID starting with 9508d3d03b271efbadeb593a4a2625d004b98263d1d961afa7a36695f5fdfe65 not found: ID does not exist"
Oct 30 11:34:29.501811 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.501786    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:29.507144 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.507118    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:29.544314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.544267    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:29.544641 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.544629    4849 cpu_manager.go:414] "containerMap: removing container" podUID="5e5b71e0-7616-4564-bdeb-f2436f2b9193" containerName="otc-container"
Oct 30 11:34:29.544692 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.544643    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e5b71e0-7616-4564-bdeb-f2436f2b9193" containerName="otc-container"
Oct 30 11:34:29.544727 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.544722    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e5b71e0-7616-4564-bdeb-f2436f2b9193" containerName="otc-container"
Oct 30 11:34:29.549248 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.549231    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.568323 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.568304    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:34:29.570268 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.570249    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:29.570381 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.570363    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:34:29.570456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.570440    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-phltc\""
Oct 30 11:34:29.570511 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.570450    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:34:29.570568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.570550    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:34:29.596316 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.596291    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.596431 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.596327    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a29b710a-9399-4d37-b994-1e58feab4d26-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.596431 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.596365    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.596536 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.596431    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.596600 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.596582    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjgf7\" (UniqueName: \"kubernetes.io/projected/a29b710a-9399-4d37-b994-1e58feab4d26-kube-api-access-qjgf7\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.697135 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.697111    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.697292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.697143    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.697292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.697249    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjgf7\" (UniqueName: \"kubernetes.io/projected/a29b710a-9399-4d37-b994-1e58feab4d26-kube-api-access-qjgf7\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.697410 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.697302    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.697410 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.697326    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a29b710a-9399-4d37-b994-1e58feab4d26-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.698001 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.697977    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a29b710a-9399-4d37-b994-1e58feab4d26-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.699761 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.699739    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.699848 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.699776    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.699891 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.699871    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.722417 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.722396    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjgf7\" (UniqueName: \"kubernetes.io/projected/a29b710a-9399-4d37-b994-1e58feab4d26-kube-api-access-qjgf7\") pod \"data-science-collector-collector-0\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:29.861229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:29.861207    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:30.161759 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.161688    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:30.164530 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:34:30.164504    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29b710a_9399_4d37_b994_1e58feab4d26.slice/crio-7b1394d9fbe75d6d6063557e77d674254d403703d059b7b4aa3d89f6c65b549e WatchSource:0}: Error finding container 7b1394d9fbe75d6d6063557e77d674254d403703d059b7b4aa3d89f6c65b549e: Status 404 returned error can't find the container with id 7b1394d9fbe75d6d6063557e77d674254d403703d059b7b4aa3d89f6c65b549e
Oct 30 11:34:30.167115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.167077    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:34:30.167210 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.167154    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:34:30.455751 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.455667    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a29b710a-9399-4d37-b994-1e58feab4d26","Type":"ContainerStarted","Data":"fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8"}
Oct 30 11:34:30.455751 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.455722    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a29b710a-9399-4d37-b994-1e58feab4d26","Type":"ContainerStarted","Data":"7b1394d9fbe75d6d6063557e77d674254d403703d059b7b4aa3d89f6c65b549e"}
Oct 30 11:34:30.502854 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.502805    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.502791818 podStartE2EDuration="1.502791818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:34:30.500716794 +0000 UTC m=+2967.881166237" watchObservedRunningTime="2026-10-30 11:34:30.502791818 +0000 UTC m=+2967.883241256"
Oct 30 11:34:30.738892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.738812    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:30.744478 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.744455    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:30.753973 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:30.753947    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e5b71e0-7616-4564-bdeb-f2436f2b9193" path="/var/lib/kubelet/pods/5e5b71e0-7616-4564-bdeb-f2436f2b9193/volumes"
Oct 30 11:34:31.465372 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:31.465344    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:39.430793 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:39.430756    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:34:41.071048 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:41.070999    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:41.071481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:41.071248    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="a29b710a-9399-4d37-b994-1e58feab4d26" containerName="otc-container" containerID="cri-o://fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8" gracePeriod=30
Oct 30 11:34:42.328977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.328951    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.398442 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.398418    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a29b710a-9399-4d37-b994-1e58feab4d26-kube-api-access-qjgf7\" (UniqueName: \"kubernetes.io/projected/a29b710a-9399-4d37-b994-1e58feab4d26-kube-api-access-qjgf7\") pod \"a29b710a-9399-4d37-b994-1e58feab4d26\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") "
Oct 30 11:34:42.398595 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.398484    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-monitoring-tls-certs\") pod \"a29b710a-9399-4d37-b994-1e58feab4d26\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") "
Oct 30 11:34:42.398595 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.398510    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-data-science-collector-ta-client-cert\") pod \"a29b710a-9399-4d37-b994-1e58feab4d26\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") "
Oct 30 11:34:42.398595 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.398573    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a29b710a-9399-4d37-b994-1e58feab4d26-otc-internal\" (UniqueName: \"kubernetes.io/configmap/a29b710a-9399-4d37-b994-1e58feab4d26-otc-internal\") pod \"a29b710a-9399-4d37-b994-1e58feab4d26\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") "
Oct 30 11:34:42.398783 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.398603    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-tls-certs\") pod \"a29b710a-9399-4d37-b994-1e58feab4d26\" (UID: \"a29b710a-9399-4d37-b994-1e58feab4d26\") "
Oct 30 11:34:42.399045 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.399000    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a29b710a-9399-4d37-b994-1e58feab4d26-otc-internal" pod "a29b710a-9399-4d37-b994-1e58feab4d26" (UID: "a29b710a-9399-4d37-b994-1e58feab4d26"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:42.400577 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.400549    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29b710a-9399-4d37-b994-1e58feab4d26-kube-api-access-qjgf7" pod "a29b710a-9399-4d37-b994-1e58feab4d26" (UID: "a29b710a-9399-4d37-b994-1e58feab4d26"). InnerVolumeSpecName "kube-api-access-qjgf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:42.401084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.401062    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-monitoring-tls-certs" pod "a29b710a-9399-4d37-b994-1e58feab4d26" (UID: "a29b710a-9399-4d37-b994-1e58feab4d26"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:42.401179 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.401122    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-tls-certs" pod "a29b710a-9399-4d37-b994-1e58feab4d26" (UID: "a29b710a-9399-4d37-b994-1e58feab4d26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:42.401179 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.401141    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-data-science-collector-ta-client-cert" pod "a29b710a-9399-4d37-b994-1e58feab4d26" (UID: "a29b710a-9399-4d37-b994-1e58feab4d26"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:42.499693 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.499670    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:42.499693 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.499690    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a29b710a-9399-4d37-b994-1e58feab4d26-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:42.499822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.499701    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:42.499822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.499711    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjgf7\" (UniqueName: \"kubernetes.io/projected/a29b710a-9399-4d37-b994-1e58feab4d26-kube-api-access-qjgf7\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:42.499822 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.499719    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a29b710a-9399-4d37-b994-1e58feab4d26-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:42.529392 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.529367    4849 generic.go:356] "Generic (PLEG): container finished" podID="a29b710a-9399-4d37-b994-1e58feab4d26" containerID="fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8" exitCode=0
Oct 30 11:34:42.529507 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.529410    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.529507 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.529460    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a29b710a-9399-4d37-b994-1e58feab4d26","Type":"ContainerDied","Data":"fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8"}
Oct 30 11:34:42.529634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.529516    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"a29b710a-9399-4d37-b994-1e58feab4d26","Type":"ContainerDied","Data":"7b1394d9fbe75d6d6063557e77d674254d403703d059b7b4aa3d89f6c65b549e"}
Oct 30 11:34:42.529634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.529550    4849 scope.go:122] "RemoveContainer" containerID="fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8"
Oct 30 11:34:42.539179 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.539164    4849 scope.go:122] "RemoveContainer" containerID="fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8"
Oct 30 11:34:42.539423 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:42.539406    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8\": container with ID starting with fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8 not found: ID does not exist" containerID="fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8"
Oct 30 11:34:42.539466 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.539431    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8"} err="failed to get container status \"fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8\": rpc error: code = NotFound desc = could not find container \"fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8\": container with ID starting with fe194488526676784d961f34ad3728a38895dec45f996fb960eb73fac24513d8 not found: ID does not exist"
Oct 30 11:34:42.584056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.584012    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:42.586564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.586547    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:42.621886 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.621864    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:42.622272 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.622257    4849 cpu_manager.go:414] "containerMap: removing container" podUID="a29b710a-9399-4d37-b994-1e58feab4d26" containerName="otc-container"
Oct 30 11:34:42.622320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.622273    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a29b710a-9399-4d37-b994-1e58feab4d26" containerName="otc-container"
Oct 30 11:34:42.622358 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.622327    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="a29b710a-9399-4d37-b994-1e58feab4d26" containerName="otc-container"
Oct 30 11:34:42.626726 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.626711    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.645257 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.645236    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:34:42.645929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.645908    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:34:42.646041 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.645952    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:34:42.646115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.646047    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:34:42.646115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.646109    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-phltc\""
Oct 30 11:34:42.647533 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.647515    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:42.701365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.701311    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.701365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.701352    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.701547 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.701374    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.701547 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.701411    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.701547 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.701518    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn4xp\" (UniqueName: \"kubernetes.io/projected/aef437af-c872-4b28-8f75-9340a6265362-kube-api-access-fn4xp\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.752904 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.752881    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a29b710a-9399-4d37-b994-1e58feab4d26" path="/var/lib/kubelet/pods/a29b710a-9399-4d37-b994-1e58feab4d26/volumes"
Oct 30 11:34:42.801967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.801946    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.802080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.801993    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.802080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.802013    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.802325 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.802303    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.802384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.802367    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn4xp\" (UniqueName: \"kubernetes.io/projected/aef437af-c872-4b28-8f75-9340a6265362-kube-api-access-fn4xp\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.802862 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.802838    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.804482 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.804460    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.804565 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.804500    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.804612 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.804568    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.834031 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.834008    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn4xp\" (UniqueName: \"kubernetes.io/projected/aef437af-c872-4b28-8f75-9340a6265362-kube-api-access-fn4xp\") pod \"data-science-collector-collector-0\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:42.939003 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:42.938983    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:43.243139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.243117    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:43.245724 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:34:43.245700    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef437af_c872_4b28_8f75_9340a6265362.slice/crio-db5adde3a74a560b89abf60812fc5036d13e52df17e382438f559f837798d830 WatchSource:0}: Error finding container db5adde3a74a560b89abf60812fc5036d13e52df17e382438f559f837798d830: Status 404 returned error can't find the container with id db5adde3a74a560b89abf60812fc5036d13e52df17e382438f559f837798d830
Oct 30 11:34:43.248341 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.248309    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:34:43.248419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.248367    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:34:43.540937 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.540841    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"aef437af-c872-4b28-8f75-9340a6265362","Type":"ContainerStarted","Data":"b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f"}
Oct 30 11:34:43.541393 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.540937    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"aef437af-c872-4b28-8f75-9340a6265362","Type":"ContainerStarted","Data":"db5adde3a74a560b89abf60812fc5036d13e52df17e382438f559f837798d830"}
Oct 30 11:34:43.587107 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.587049    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.587017451 podStartE2EDuration="1.587017451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:34:43.586247561 +0000 UTC m=+2980.966697001" watchObservedRunningTime="2026-10-30 11:34:43.587017451 +0000 UTC m=+2980.967466892"
Oct 30 11:34:43.766704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.766670    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:34:43.767071 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.766998    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="prometheus" containerID="cri-o://01e16fd02a9d3708367f021dd22a1c59372672952cb6f46135775d34cf4f27e8" gracePeriod=600
Oct 30 11:34:43.767226 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.767096    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="thanos-sidecar" containerID="cri-o://ac48755d426db1f7f224406cd0128d6b8fcb70057b969875c8ebb7acb2c16211" gracePeriod=600
Oct 30 11:34:43.767516 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.767123    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="config-reloader" containerID="cri-o://cc3f8e4cdee469e8e376575a980757458e156380f0865560d6f1f43b0eb9cda9" gracePeriod=600
Oct 30 11:34:43.786803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.786760    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:43.787089 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.787063    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="alertmanager" containerID="cri-o://bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64" gracePeriod=120
Oct 30 11:34:43.787908 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:43.787879    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="config-reloader" containerID="cri-o://4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565" gracePeriod=120
Oct 30 11:34:44.404771 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.404742    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:44.555735 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.555686    4849 generic.go:356] "Generic (PLEG): container finished" podID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerID="ac48755d426db1f7f224406cd0128d6b8fcb70057b969875c8ebb7acb2c16211" exitCode=0
Oct 30 11:34:44.555735 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.555722    4849 generic.go:356] "Generic (PLEG): container finished" podID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerID="cc3f8e4cdee469e8e376575a980757458e156380f0865560d6f1f43b0eb9cda9" exitCode=0
Oct 30 11:34:44.555735 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.555732    4849 generic.go:356] "Generic (PLEG): container finished" podID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerID="01e16fd02a9d3708367f021dd22a1c59372672952cb6f46135775d34cf4f27e8" exitCode=0
Oct 30 11:34:44.556197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.555824    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerDied","Data":"ac48755d426db1f7f224406cd0128d6b8fcb70057b969875c8ebb7acb2c16211"}
Oct 30 11:34:44.556197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.555890    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerDied","Data":"cc3f8e4cdee469e8e376575a980757458e156380f0865560d6f1f43b0eb9cda9"}
Oct 30 11:34:44.556197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.555931    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerDied","Data":"01e16fd02a9d3708367f021dd22a1c59372672952cb6f46135775d34cf4f27e8"}
Oct 30 11:34:44.558414 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.558385    4849 generic.go:356] "Generic (PLEG): container finished" podID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerID="4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565" exitCode=0
Oct 30 11:34:44.558543 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.558451    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerDied","Data":"4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565"}
Oct 30 11:34:44.558736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.558716    4849 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="redhat-ods-monitoring/data-science-collector-collector-0" secret="" err="secret \"data-science-collector-collector-dockercfg-phltc\" not found"
Oct 30 11:34:44.620189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.620163    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:44.620375 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:44.620352    4849 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-collector-collector-e4cadd38: configmap "data-science-collector-collector-e4cadd38" not found
Oct 30 11:34:44.620458 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:44.620439    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal podName:aef437af-c872-4b28-8f75-9340a6265362 nodeName:}" failed. No retries permitted until 2026-10-30 11:34:45.120421479 +0000 UTC m=+2982.500870914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "otc-internal" (UniqueName: "kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal") pod "data-science-collector-collector-0" (UID: "aef437af-c872-4b28-8f75-9340a6265362") : configmap "data-science-collector-collector-e4cadd38" not found
Oct 30 11:34:44.720753 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.720679    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.720896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.720748    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.720976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.720915    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721067 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.720973    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721067 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721042    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-tls-assets\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-tls-assets\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721192 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721057    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-1" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:44.721192 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721077    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721192 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721140    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721197    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-secret-prometheus-web-tls-ca\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721213    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-0" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:44.721337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721231    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721261    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-kube-api-access-hdzg6\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-kube-api-access-hdzg6\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721300    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config-out\" (UniqueName: \"kubernetes.io/empty-dir/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config-out\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721337 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721331    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721368    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-thanos-prometheus-http-client-file\") pod \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\" (UID: \"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf\") "
Oct 30 11:34:44.721689 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721670    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.721745 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.721689    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.722379 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.722352    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-2" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:44.724631 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.724356    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-thanos-prometheus-http-client-file" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:44.724786 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.724754    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-tls-assets" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:44.724971 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.724921    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:44.725583 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.725560    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config-out" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:34:44.726458 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.726417    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:44.726659 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.726623    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:44.726760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.726680    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-kube-api-access-hdzg6" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "kube-api-access-hdzg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:44.727000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.726977    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-secret-prometheus-web-tls-ca" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:44.727365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.727346    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:34:44.738362 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.738342    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config" pod "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" (UID: "d65e9a68-3e4f-42b8-986e-ac3163a2ecdf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:44.823116 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823091    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823116 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823115    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823125    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823133    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823143    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823154    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823163    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdzg6\" (UniqueName: \"kubernetes.io/projected/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-kube-api-access-hdzg6\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823171    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823180    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823189    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:44.823264 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.823214    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:34:44.834163 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.834146    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:34:44.923677 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:44.923652    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\""
Oct 30 11:34:45.125601 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:45.125577    4849 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-collector-collector-e4cadd38: configmap "data-science-collector-collector-e4cadd38" not found
Oct 30 11:34:45.125708 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:45.125643    4849 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal podName:aef437af-c872-4b28-8f75-9340a6265362 nodeName:}" failed. No retries permitted until 2026-10-30 11:34:46.125624307 +0000 UTC m=+2983.506073736 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "otc-internal" (UniqueName: "kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal") pod "data-science-collector-collector-0" (UID: "aef437af-c872-4b28-8f75-9340a6265362") : configmap "data-science-collector-collector-e4cadd38" not found
Oct 30 11:34:45.395603 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.395521    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.122:9093/-/ready\": dial tcp 10.132.0.122:9093: connect: connection refused"
Oct 30 11:34:45.566132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.566096    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"d65e9a68-3e4f-42b8-986e-ac3163a2ecdf","Type":"ContainerDied","Data":"5d0cd899c13126734847b60e139ded82a26c232d543d5e768fe653900c6b46ab"}
Oct 30 11:34:45.566132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.566120    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:34:45.566597 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.566172    4849 scope.go:122] "RemoveContainer" containerID="ac48755d426db1f7f224406cd0128d6b8fcb70057b969875c8ebb7acb2c16211"
Oct 30 11:34:45.566597 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.566454    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="aef437af-c872-4b28-8f75-9340a6265362" containerName="otc-container" containerID="cri-o://b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f" gracePeriod=30
Oct 30 11:34:45.574910 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.574893    4849 scope.go:122] "RemoveContainer" containerID="cc3f8e4cdee469e8e376575a980757458e156380f0865560d6f1f43b0eb9cda9"
Oct 30 11:34:45.630741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.630725    4849 scope.go:122] "RemoveContainer" containerID="01e16fd02a9d3708367f021dd22a1c59372672952cb6f46135775d34cf4f27e8"
Oct 30 11:34:45.668415 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.668359    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:34:45.677819 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.677794    4849 scope.go:122] "RemoveContainer" containerID="7bc379f2f495a05f57de3b140c3a24456a2ff0cbf488567e50154347198d4e72"
Oct 30 11:34:45.682173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.682148    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:34:45.816850 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.816830    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:45.932808 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.932749    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal\" (UniqueName: \"kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal\") pod \"aef437af-c872-4b28-8f75-9340a6265362\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") "
Oct 30 11:34:45.932808 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.932798    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-data-science-collector-ta-client-cert\") pod \"aef437af-c872-4b28-8f75-9340a6265362\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") "
Oct 30 11:34:45.932992 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.932841    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-tls-certs\") pod \"aef437af-c872-4b28-8f75-9340a6265362\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") "
Oct 30 11:34:45.932992 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.932867    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aef437af-c872-4b28-8f75-9340a6265362-kube-api-access-fn4xp\" (UniqueName: \"kubernetes.io/projected/aef437af-c872-4b28-8f75-9340a6265362-kube-api-access-fn4xp\") pod \"aef437af-c872-4b28-8f75-9340a6265362\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") "
Oct 30 11:34:45.932992 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.932920    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-monitoring-tls-certs\") pod \"aef437af-c872-4b28-8f75-9340a6265362\" (UID: \"aef437af-c872-4b28-8f75-9340a6265362\") "
Oct 30 11:34:45.933204 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.933173    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal" pod "aef437af-c872-4b28-8f75-9340a6265362" (UID: "aef437af-c872-4b28-8f75-9340a6265362"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:34:45.934989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.934958    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef437af-c872-4b28-8f75-9340a6265362-kube-api-access-fn4xp" pod "aef437af-c872-4b28-8f75-9340a6265362" (UID: "aef437af-c872-4b28-8f75-9340a6265362"). InnerVolumeSpecName "kube-api-access-fn4xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:45.935136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.935069    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-data-science-collector-ta-client-cert" pod "aef437af-c872-4b28-8f75-9340a6265362" (UID: "aef437af-c872-4b28-8f75-9340a6265362"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:45.935136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.935102    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-monitoring-tls-certs" pod "aef437af-c872-4b28-8f75-9340a6265362" (UID: "aef437af-c872-4b28-8f75-9340a6265362"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:45.935136 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:45.935116    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-tls-certs" pod "aef437af-c872-4b28-8f75-9340a6265362" (UID: "aef437af-c872-4b28-8f75-9340a6265362"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:46.034158 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.034137    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:46.034158 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.034157    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/aef437af-c872-4b28-8f75-9340a6265362-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:46.034287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.034166    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:46.034287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.034176    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef437af-c872-4b28-8f75-9340a6265362-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:46.034287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.034184    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn4xp\" (UniqueName: \"kubernetes.io/projected/aef437af-c872-4b28-8f75-9340a6265362-kube-api-access-fn4xp\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:46.573849 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.573817    4849 generic.go:356] "Generic (PLEG): container finished" podID="aef437af-c872-4b28-8f75-9340a6265362" containerID="b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f" exitCode=0
Oct 30 11:34:46.574288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.573870    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:34:46.574288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.573910    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"aef437af-c872-4b28-8f75-9340a6265362","Type":"ContainerDied","Data":"b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f"}
Oct 30 11:34:46.574288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.573987    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"aef437af-c872-4b28-8f75-9340a6265362","Type":"ContainerDied","Data":"db5adde3a74a560b89abf60812fc5036d13e52df17e382438f559f837798d830"}
Oct 30 11:34:46.574288 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.574048    4849 scope.go:122] "RemoveContainer" containerID="b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f"
Oct 30 11:34:46.584098 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.584077    4849 scope.go:122] "RemoveContainer" containerID="b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f"
Oct 30 11:34:46.584374 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:46.584353    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f\": container with ID starting with b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f not found: ID does not exist" containerID="b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f"
Oct 30 11:34:46.584416 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.584382    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f"} err="failed to get container status \"b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f\": rpc error: code = NotFound desc = could not find container \"b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f\": container with ID starting with b73c0f7944ca1eac9f9e849fa3c6aaf5fdebab7503f0512de5a8bb3529cc694f not found: ID does not exist"
Oct 30 11:34:46.630309 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.630288    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:46.643608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.643583    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:34:46.758331 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.758307    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aef437af-c872-4b28-8f75-9340a6265362" path="/var/lib/kubelet/pods/aef437af-c872-4b28-8f75-9340a6265362/volumes"
Oct 30 11:34:46.758736 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:46.758724    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" path="/var/lib/kubelet/pods/d65e9a68-3e4f-42b8-986e-ac3163a2ecdf/volumes"
Oct 30 11:34:50.395434 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:50.395393    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.122:9093/-/ready\": dial tcp 10.132.0.122:9093: connect: connection refused"
Oct 30 11:34:54.035197 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.035173    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:54.089575 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089548    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-alertmanager-data-science-monitoringstack-db\") pod \"29bc7634-538b-44d8-8620-edb1fa360d7e\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") "
Oct 30 11:34:54.089717 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089591    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-config-out\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-config-out\") pod \"29bc7634-538b-44d8-8620-edb1fa360d7e\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") "
Oct 30 11:34:54.089717 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089643    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-kube-api-access-zzzbk\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-kube-api-access-zzzbk\") pod \"29bc7634-538b-44d8-8620-edb1fa360d7e\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") "
Oct 30 11:34:54.089717 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089711    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-cluster-tls-config\") pod \"29bc7634-538b-44d8-8620-edb1fa360d7e\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") "
Oct 30 11:34:54.089893 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089750    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-web-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-web-config\") pod \"29bc7634-538b-44d8-8620-edb1fa360d7e\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") "
Oct 30 11:34:54.089893 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089787    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-config-volume\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-config-volume\") pod \"29bc7634-538b-44d8-8620-edb1fa360d7e\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") "
Oct 30 11:34:54.089893 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089821    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-tls-assets\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-tls-assets\") pod \"29bc7634-538b-44d8-8620-edb1fa360d7e\" (UID: \"29bc7634-538b-44d8-8620-edb1fa360d7e\") "
Oct 30 11:34:54.089893 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.089875    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-alertmanager-data-science-monitoringstack-db" pod "29bc7634-538b-44d8-8620-edb1fa360d7e" (UID: "29bc7634-538b-44d8-8620-edb1fa360d7e"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:34:54.090125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.090096    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:54.091998 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.091971    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-tls-assets" pod "29bc7634-538b-44d8-8620-edb1fa360d7e" (UID: "29bc7634-538b-44d8-8620-edb1fa360d7e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:54.092294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.092269    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-config-out" pod "29bc7634-538b-44d8-8620-edb1fa360d7e" (UID: "29bc7634-538b-44d8-8620-edb1fa360d7e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:34:54.092376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.092328    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-kube-api-access-zzzbk" pod "29bc7634-538b-44d8-8620-edb1fa360d7e" (UID: "29bc7634-538b-44d8-8620-edb1fa360d7e"). InnerVolumeSpecName "kube-api-access-zzzbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:34:54.092376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.092355    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-config-volume" pod "29bc7634-538b-44d8-8620-edb1fa360d7e" (UID: "29bc7634-538b-44d8-8620-edb1fa360d7e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:54.095990 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.095969    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-cluster-tls-config" pod "29bc7634-538b-44d8-8620-edb1fa360d7e" (UID: "29bc7634-538b-44d8-8620-edb1fa360d7e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:54.103034 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.102998    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-web-config" pod "29bc7634-538b-44d8-8620-edb1fa360d7e" (UID: "29bc7634-538b-44d8-8620-edb1fa360d7e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:34:54.191437 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.191409    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzzbk\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-kube-api-access-zzzbk\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:54.191437 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.191435    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:54.191589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.191448    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:54.191589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.191461    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29bc7634-538b-44d8-8620-edb1fa360d7e-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:54.191589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.191472    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29bc7634-538b-44d8-8620-edb1fa360d7e-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:54.191589 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.191483    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29bc7634-538b-44d8-8620-edb1fa360d7e-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:34:54.620814 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.620781    4849 generic.go:356] "Generic (PLEG): container finished" podID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerID="bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64" exitCode=0
Oct 30 11:34:54.620951 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.620866    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:34:54.620951 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.620872    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerDied","Data":"bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64"}
Oct 30 11:34:54.621063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.620956    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"29bc7634-538b-44d8-8620-edb1fa360d7e","Type":"ContainerDied","Data":"9aefe06267d4dcac7f217eaf16cd12f6363dbd1f7f68e611596c3a0ba32de9bb"}
Oct 30 11:34:54.621063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.620998    4849 scope.go:122] "RemoveContainer" containerID="4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565"
Oct 30 11:34:54.631004 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.630982    4849 scope.go:122] "RemoveContainer" containerID="bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64"
Oct 30 11:34:54.640078 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.640048    4849 scope.go:122] "RemoveContainer" containerID="bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3"
Oct 30 11:34:54.648546 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.648524    4849 scope.go:122] "RemoveContainer" containerID="4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565"
Oct 30 11:34:54.648815 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:54.648790    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565\": container with ID starting with 4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565 not found: ID does not exist" containerID="4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565"
Oct 30 11:34:54.648875 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.648820    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565"} err="failed to get container status \"4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565\": rpc error: code = NotFound desc = could not find container \"4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565\": container with ID starting with 4dd4ffb071c05db0f75b29a9ae5343ecf82319a24747a8ae73793bc6a75d7565 not found: ID does not exist"
Oct 30 11:34:54.648875 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.648839    4849 scope.go:122] "RemoveContainer" containerID="bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64"
Oct 30 11:34:54.649061 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:54.649042    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64\": container with ID starting with bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64 not found: ID does not exist" containerID="bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64"
Oct 30 11:34:54.649107 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.649065    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64"} err="failed to get container status \"bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64\": rpc error: code = NotFound desc = could not find container \"bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64\": container with ID starting with bd33a696833fdef918969219ab83c9efc9f966da29f6268dc825fc8c8f992c64 not found: ID does not exist"
Oct 30 11:34:54.649107 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.649078    4849 scope.go:122] "RemoveContainer" containerID="bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3"
Oct 30 11:34:54.649288 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:34:54.649272    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3\": container with ID starting with bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3 not found: ID does not exist" containerID="bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3"
Oct 30 11:34:54.649332 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.649290    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3"} err="failed to get container status \"bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3\": rpc error: code = NotFound desc = could not find container \"bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3\": container with ID starting with bc2386d847222dc8ea765d15c254106b3a39933c5e36086e67fcbd6a1ebafdb3 not found: ID does not exist"
Oct 30 11:34:54.692146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.692123    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:54.705585 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.705561    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:34:54.752513 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:34:54.752478    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" path="/var/lib/kubelet/pods/29bc7634-538b-44d8-8620-edb1fa360d7e/volumes"
Oct 30 11:35:01.932794 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.932758    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933146    4849 cpu_manager.go:414] "containerMap: removing container" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="alertmanager"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933159    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="alertmanager"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933167    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="thanos-sidecar"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933172    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="thanos-sidecar"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933197    4849 cpu_manager.go:414] "containerMap: removing container" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933203    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933210    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="prometheus"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933214    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="prometheus"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933221    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933226    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933238    4849 cpu_manager.go:414] "containerMap: removing container" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="init-config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933243    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="init-config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933249    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="init-config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933253    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="init-config-reloader"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933262    4849 cpu_manager.go:414] "containerMap: removing container" podUID="aef437af-c872-4b28-8f75-9340a6265362" containerName="otc-container"
Oct 30 11:35:01.933291 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933266    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aef437af-c872-4b28-8f75-9340a6265362" containerName="otc-container"
Oct 30 11:35:01.934084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933317    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="aef437af-c872-4b28-8f75-9340a6265362" containerName="otc-container"
Oct 30 11:35:01.934084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933325    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="alertmanager"
Oct 30 11:35:01.934084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933330    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="thanos-sidecar"
Oct 30 11:35:01.934084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933341    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="29bc7634-538b-44d8-8620-edb1fa360d7e" containerName="config-reloader"
Oct 30 11:35:01.934084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933349    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="prometheus"
Oct 30 11:35:01.934084 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.933357    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d65e9a68-3e4f-42b8-986e-ac3163a2ecdf" containerName="config-reloader"
Oct 30 11:35:01.937976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.937959    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:01.986826 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.986796    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:35:01.988211 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:01.988186    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:35:02.000765 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.000738    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:35:02.001939 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.001916    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:35:02.002093 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.001943    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-7p6tb\""
Oct 30 11:35:02.002250 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.002224    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:35:02.002345 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.002296    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:35:02.002579 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.002558    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:02.047676 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.047647    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.047803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.047684    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4clf\" (UniqueName: \"kubernetes.io/projected/f694f17d-527a-4a91-928e-de313fcd6721-kube-api-access-x4clf\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.047873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.047807    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f694f17d-527a-4a91-928e-de313fcd6721-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.047873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.047841    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.047983 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.047905    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.149221 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.149191    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.149409 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.149248    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.149409 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.149268    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4clf\" (UniqueName: \"kubernetes.io/projected/f694f17d-527a-4a91-928e-de313fcd6721-kube-api-access-x4clf\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.149409 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.149298    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f694f17d-527a-4a91-928e-de313fcd6721-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.149409 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.149323    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.150109 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.150085    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f694f17d-527a-4a91-928e-de313fcd6721-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.152062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.152013    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.152203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.152175    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.152255 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.152219    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.177419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.177392    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4clf\" (UniqueName: \"kubernetes.io/projected/f694f17d-527a-4a91-928e-de313fcd6721-kube-api-access-x4clf\") pod \"data-science-collector-collector-0\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.249894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.249835    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:02.555979 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.555943    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:02.558246 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:35:02.558217    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf694f17d_527a_4a91_928e_de313fcd6721.slice/crio-85bf1eda2d38367133c6dd3d290b64fa27d4b98a7f4550fe1f859a69e125cca8 WatchSource:0}: Error finding container 85bf1eda2d38367133c6dd3d290b64fa27d4b98a7f4550fe1f859a69e125cca8: Status 404 returned error can't find the container with id 85bf1eda2d38367133c6dd3d290b64fa27d4b98a7f4550fe1f859a69e125cca8
Oct 30 11:35:02.560798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.560759    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:35:02.560895 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.560844    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:35:02.665184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.665140    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f694f17d-527a-4a91-928e-de313fcd6721","Type":"ContainerStarted","Data":"0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46"}
Oct 30 11:35:02.665344 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.665211    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f694f17d-527a-4a91-928e-de313fcd6721","Type":"ContainerStarted","Data":"85bf1eda2d38367133c6dd3d290b64fa27d4b98a7f4550fe1f859a69e125cca8"}
Oct 30 11:35:02.711296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:02.711246    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.711232149 podStartE2EDuration="1.711232149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:35:02.710004087 +0000 UTC m=+3000.090453530" watchObservedRunningTime="2026-10-30 11:35:02.711232149 +0000 UTC m=+3000.091681587"
Oct 30 11:35:03.085969 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:03.085943    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:35:03.103891 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:03.103856    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:35:05.351119 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.351087    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:05.356695 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.356674    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.377103 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.377071    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:05.378828 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.378802    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:35:05.382465 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.382446    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.387421 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.387385    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 11:35:05.387530 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.387424    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 11:35:05.387601 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.387524    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 11:35:05.408666 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.408634    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:35:05.464050 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464010    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 11:35:05.464194 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464068    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 11:35:05.464266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464211    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 11:35:05.464266 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464212    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:35:05.464391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464289    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 11:35:05.464391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464294    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 11:35:05.464391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464296    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:35:05.464391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464318    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-n5xr8\""
Oct 30 11:35:05.464391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464325    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 11:35:05.464391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464372    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 11:35:05.464608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464406    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-kh4cb\""
Oct 30 11:35:05.464608 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.464452    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 11:35:05.478074 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.477999    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.478169 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478084    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478169 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478115    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-tls-assets\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.478169 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478138    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478280 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478188    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcsch\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-kube-api-access-qcsch\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478280 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478251    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-volume\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.478280 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478272    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-out\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.478378 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478299    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478378 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478342    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478378 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478359    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478377    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478404    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478504 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478421    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478631 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478530    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478631 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478597    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-web-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.478707 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478636    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.478707 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478660    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478717    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.478799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.478735    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92sww\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-kube-api-access-92sww\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.579464 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579437    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.579561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579468    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.579561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579487    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.579561 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579505    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-web-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.579701 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579654    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.579760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579711    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.579760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579754    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.579873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579784    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92sww\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-kube-api-access-92sww\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.579873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579817    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.579873 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579846    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.580059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579879    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-tls-assets\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.580059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579917    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.580059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579943    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcsch\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-kube-api-access-qcsch\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.580059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579973    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-volume\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.580059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.579999    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-out\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.580059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.580055    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.580398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.580065    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.580398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.580117    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.580398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.580150    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.580398 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.580180    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.580722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.580696    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.581173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.581119    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.582881 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.582851    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.582975 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.582919    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-web-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.583137 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.583117    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-tls-assets\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.583669 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.583642    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.585378 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.584296    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.585378 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.585322    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-out\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.585378 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.585336    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-volume\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.585562 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.585432    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.585903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.585831    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.586299 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.586276    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.586568 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.586546    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.586623 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.586548    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.586750 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.586733    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.586786 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.586729    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.608296 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.608247    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcsch\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-kube-api-access-qcsch\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:05.608989 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.608971    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92sww\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-kube-api-access-92sww\") pod \"alertmanager-data-science-monitoringstack-1\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:05.699835 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:05.699811    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:06.032765 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:06.032739    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:35:06.034807 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:35:06.034777    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ce63f8_5e15_479a_912a_c2fee9cbaa10.slice/crio-9e3af703e7dcdbdfb944a192a7dba5795622ef5675640974a69e59e32d5d179d WatchSource:0}: Error finding container 9e3af703e7dcdbdfb944a192a7dba5795622ef5675640974a69e59e32d5d179d: Status 404 returned error can't find the container with id 9e3af703e7dcdbdfb944a192a7dba5795622ef5675640974a69e59e32d5d179d
Oct 30 11:35:06.685481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:06.685413    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerStarted","Data":"bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00"}
Oct 30 11:35:06.685481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:06.685472    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerStarted","Data":"9e3af703e7dcdbdfb944a192a7dba5795622ef5675640974a69e59e32d5d179d"}
Oct 30 11:35:07.801799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:07.801765    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:07.833463 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:07.833426    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:07.902336 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:07.902309    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:07.902433 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:07.902416    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:07.918009 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:07.917988    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:08.020170 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:08.020147    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5f4e377c3e259c071aab429e31ca61846b879cc3476e2f02b70e888ccdcdb590/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:08.048234 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:08.048214    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:08.078010 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:08.077986    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:08.413305 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:08.413281    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:08.414850 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:35:08.414822    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc491ab1_295c_4f88_b744_80ca1ac1dedd.slice/crio-d7f183e5f11628a418a5ed94c91f19fbc86f98a03d1eb399da8083f0785050e6 WatchSource:0}: Error finding container d7f183e5f11628a418a5ed94c91f19fbc86f98a03d1eb399da8083f0785050e6: Status 404 returned error can't find the container with id d7f183e5f11628a418a5ed94c91f19fbc86f98a03d1eb399da8083f0785050e6
Oct 30 11:35:08.696929 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:08.696870    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerStarted","Data":"d7f183e5f11628a418a5ed94c91f19fbc86f98a03d1eb399da8083f0785050e6"}
Oct 30 11:35:09.431649 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:09.431614    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:35:09.703384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:09.703329    4849 generic.go:356] "Generic (PLEG): container finished" podID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerID="bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00" exitCode=0
Oct 30 11:35:09.703503 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:09.703413    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerDied","Data":"bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00"}
Oct 30 11:35:09.705203 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:09.705166    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerStarted","Data":"c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b"}
Oct 30 11:35:10.713156 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:10.713120    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerStarted","Data":"f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332"}
Oct 30 11:35:10.713551 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:10.713190    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerStarted","Data":"d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225"}
Oct 30 11:35:10.713551 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:10.713470    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:10.791383 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:10.791335    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podStartSLOduration=2.506577692 podStartE2EDuration="5.791321527s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.284743835s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:35:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:35:10.7763942 +0000 UTC m=+3008.156843638" watchObservedRunningTime="2026-10-30 11:35:10.791321527 +0000 UTC m=+3008.171770999"
Oct 30 11:35:12.724454 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:12.724422    4849 generic.go:356] "Generic (PLEG): container finished" podID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerID="c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b" exitCode=0
Oct 30 11:35:12.724807 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:12.724493    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerDied","Data":"c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b"}
Oct 30 11:35:13.732005 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:13.731965    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerStarted","Data":"c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da"}
Oct 30 11:35:13.732410 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:13.732062    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerStarted","Data":"754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d"}
Oct 30 11:35:13.732410 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:13.732090    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerStarted","Data":"7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54"}
Oct 30 11:35:13.817949 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:13.817905    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podStartSLOduration=5.432600562 podStartE2EDuration="8.817893563s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.385293001s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:35:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:35:13.802926137 +0000 UTC m=+3011.183375577" watchObservedRunningTime="2026-10-30 11:35:13.817893563 +0000 UTC m=+3011.198343003"
Oct 30 11:35:16.719912 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:16.719883    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:18.079104 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:18.079074    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:23.078803 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.078763    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:23.083815 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.083792    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:23.323531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.323494    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:23.323805 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.323745    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="f694f17d-527a-4a91-928e-de313fcd6721" containerName="otc-container" containerID="cri-o://0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46" gracePeriod=30
Oct 30 11:35:23.568818 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.568796    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:23.730463 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.730397    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-data-science-collector-ta-client-cert\") pod \"f694f17d-527a-4a91-928e-de313fcd6721\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") "
Oct 30 11:35:23.730463 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.730451    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-monitoring-tls-certs\") pod \"f694f17d-527a-4a91-928e-de313fcd6721\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") "
Oct 30 11:35:23.730649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.730483    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f694f17d-527a-4a91-928e-de313fcd6721-otc-internal\" (UniqueName: \"kubernetes.io/configmap/f694f17d-527a-4a91-928e-de313fcd6721-otc-internal\") pod \"f694f17d-527a-4a91-928e-de313fcd6721\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") "
Oct 30 11:35:23.730649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.730512    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f694f17d-527a-4a91-928e-de313fcd6721-kube-api-access-x4clf\" (UniqueName: \"kubernetes.io/projected/f694f17d-527a-4a91-928e-de313fcd6721-kube-api-access-x4clf\") pod \"f694f17d-527a-4a91-928e-de313fcd6721\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") "
Oct 30 11:35:23.730649 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.730599    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-tls-certs\") pod \"f694f17d-527a-4a91-928e-de313fcd6721\" (UID: \"f694f17d-527a-4a91-928e-de313fcd6721\") "
Oct 30 11:35:23.730880 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.730844    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f694f17d-527a-4a91-928e-de313fcd6721-otc-internal" pod "f694f17d-527a-4a91-928e-de313fcd6721" (UID: "f694f17d-527a-4a91-928e-de313fcd6721"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:35:23.732656 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.732627    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-tls-certs" pod "f694f17d-527a-4a91-928e-de313fcd6721" (UID: "f694f17d-527a-4a91-928e-de313fcd6721"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:23.732776 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.732678    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-monitoring-tls-certs" pod "f694f17d-527a-4a91-928e-de313fcd6721" (UID: "f694f17d-527a-4a91-928e-de313fcd6721"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:23.732836 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.732767    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-data-science-collector-ta-client-cert" pod "f694f17d-527a-4a91-928e-de313fcd6721" (UID: "f694f17d-527a-4a91-928e-de313fcd6721"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:23.732836 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.732779    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f694f17d-527a-4a91-928e-de313fcd6721-kube-api-access-x4clf" pod "f694f17d-527a-4a91-928e-de313fcd6721" (UID: "f694f17d-527a-4a91-928e-de313fcd6721"). InnerVolumeSpecName "kube-api-access-x4clf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:35:23.784488 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.784463    4849 generic.go:356] "Generic (PLEG): container finished" podID="f694f17d-527a-4a91-928e-de313fcd6721" containerID="0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46" exitCode=0
Oct 30 11:35:23.784606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.784508    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:23.784606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.784551    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f694f17d-527a-4a91-928e-de313fcd6721","Type":"ContainerDied","Data":"0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46"}
Oct 30 11:35:23.784606 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.784601    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"f694f17d-527a-4a91-928e-de313fcd6721","Type":"ContainerDied","Data":"85bf1eda2d38367133c6dd3d290b64fa27d4b98a7f4550fe1f859a69e125cca8"}
Oct 30 11:35:23.784773 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.784631    4849 scope.go:122] "RemoveContainer" containerID="0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46"
Oct 30 11:35:23.790862 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.790846    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:23.793679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.793630    4849 scope.go:122] "RemoveContainer" containerID="0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46"
Oct 30 11:35:23.793971 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:23.793950    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46\": container with ID starting with 0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46 not found: ID does not exist" containerID="0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46"
Oct 30 11:35:23.794094 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.793977    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46"} err="failed to get container status \"0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46\": rpc error: code = NotFound desc = could not find container \"0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46\": container with ID starting with 0415b41d6a25a9ac8055dd7f240aceaefbd538c24460b94e80d5571e1b7e3c46 not found: ID does not exist"
Oct 30 11:35:23.831747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.831724    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:23.831747 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.831747    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:23.831868 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.831756    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f694f17d-527a-4a91-928e-de313fcd6721-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:23.831868 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.831765    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f694f17d-527a-4a91-928e-de313fcd6721-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:23.831868 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.831773    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4clf\" (UniqueName: \"kubernetes.io/projected/f694f17d-527a-4a91-928e-de313fcd6721-kube-api-access-x4clf\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:23.839129 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.839110    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:23.846259 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.846236    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:23.883620 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.883600    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:23.884586 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.884554    4849 cpu_manager.go:414] "containerMap: removing container" podUID="f694f17d-527a-4a91-928e-de313fcd6721" containerName="otc-container"
Oct 30 11:35:23.884586 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.884579    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f694f17d-527a-4a91-928e-de313fcd6721" containerName="otc-container"
Oct 30 11:35:23.884837 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.884814    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="f694f17d-527a-4a91-928e-de313fcd6721" containerName="otc-container"
Oct 30 11:35:23.892121 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.892105    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:23.908778 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.908757    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-7p6tb\""
Oct 30 11:35:23.909175 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.909155    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:23.911309 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.911290    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:35:23.911967 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:23.911946    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:35:24.008853 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.008788    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:35:24.008853 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.008806    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:35:24.033634 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.033612    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.033746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.033654    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.033746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.033687    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.033817 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.033769    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vswgt\" (UniqueName: \"kubernetes.io/projected/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-kube-api-access-vswgt\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.033817 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.033806    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.134535 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.134510    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.134932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.134545    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vswgt\" (UniqueName: \"kubernetes.io/projected/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-kube-api-access-vswgt\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.134932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.134570    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.134932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.134593    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.134932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.134628    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.135374 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.135345    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.136937 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.136912    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.137057 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.137005    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.137125 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.137055    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.160844 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.160822    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vswgt\" (UniqueName: \"kubernetes.io/projected/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-kube-api-access-vswgt\") pod \"data-science-collector-collector-0\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.205111 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.205094    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:24.504126 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.504102    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:24.506430 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:35:24.506403    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8916c0f0_5e82_418d_ad04_5e4ab5a3ae5e.slice/crio-6d19abdf7d992dbbe1e0e7ac94b87450d2dc1cb76c2d088e32d5b285be4a0d93 WatchSource:0}: Error finding container 6d19abdf7d992dbbe1e0e7ac94b87450d2dc1cb76c2d088e32d5b285be4a0d93: Status 404 returned error can't find the container with id 6d19abdf7d992dbbe1e0e7ac94b87450d2dc1cb76c2d088e32d5b285be4a0d93
Oct 30 11:35:24.508549 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.508516    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:35:24.508654 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.508585    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:35:24.753950 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.753869    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f694f17d-527a-4a91-928e-de313fcd6721" path="/var/lib/kubelet/pods/f694f17d-527a-4a91-928e-de313fcd6721/volumes"
Oct 30 11:35:24.790739 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.790704    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e","Type":"ContainerStarted","Data":"5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c"}
Oct 30 11:35:24.790978 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.790761    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e","Type":"ContainerStarted","Data":"6d19abdf7d992dbbe1e0e7ac94b87450d2dc1cb76c2d088e32d5b285be4a0d93"}
Oct 30 11:35:24.837419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:24.837363    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.837349699 podStartE2EDuration="1.837349699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:35:24.834693063 +0000 UTC m=+3022.215142501" watchObservedRunningTime="2026-10-30 11:35:24.837349699 +0000 UTC m=+3022.217799209"
Oct 30 11:35:35.898168 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:35.898131    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:35.898595 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:35.898404    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" containerName="otc-container" containerID="cri-o://5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c" gracePeriod=30
Oct 30 11:35:36.150907 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.150852    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:36.229153 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.229116    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-otc-internal\" (UniqueName: \"kubernetes.io/configmap/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-otc-internal\") pod \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") "
Oct 30 11:35:36.229153 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.229163    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-tls-certs\") pod \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") "
Oct 30 11:35:36.229391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.229189    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-monitoring-tls-certs\") pod \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") "
Oct 30 11:35:36.229391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.229218    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-data-science-collector-ta-client-cert\") pod \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") "
Oct 30 11:35:36.229391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.229292    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-kube-api-access-vswgt\" (UniqueName: \"kubernetes.io/projected/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-kube-api-access-vswgt\") pod \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\" (UID: \"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e\") "
Oct 30 11:35:36.229576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.229551    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-otc-internal" pod "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" (UID: "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:35:36.229682 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.229666    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:36.231384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.231350    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-tls-certs" pod "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" (UID: "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:36.231384 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.231365    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-monitoring-tls-certs" pod "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" (UID: "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:36.231532 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.231417    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-data-science-collector-ta-client-cert" pod "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" (UID: "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:36.231532 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.231428    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-kube-api-access-vswgt" pod "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" (UID: "8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e"). InnerVolumeSpecName "kube-api-access-vswgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:35:36.330396 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.330372    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:36.330396 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.330392    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:36.330542 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.330403    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:36.330542 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.330413    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vswgt\" (UniqueName: \"kubernetes.io/projected/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e-kube-api-access-vswgt\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:36.580583 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.580549    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:35:36.580830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.580802    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="alertmanager" containerID="cri-o://d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225" gracePeriod=120
Oct 30 11:35:36.581056 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.580851    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="config-reloader" containerID="cri-o://f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332" gracePeriod=120
Oct 30 11:35:36.592711 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.592647    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:36.593105 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.593074    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="prometheus" containerID="cri-o://7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54" gracePeriod=600
Oct 30 11:35:36.593347 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.593099    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="thanos-sidecar" containerID="cri-o://c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da" gracePeriod=600
Oct 30 11:35:36.593481 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.593107    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="config-reloader" containerID="cri-o://754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d" gracePeriod=600
Oct 30 11:35:36.718592 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.718558    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.127:9093/-/ready\": dial tcp 10.132.0.127:9093: connect: connection refused"
Oct 30 11:35:36.856059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.855981    4849 generic.go:356] "Generic (PLEG): container finished" podID="8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" containerID="5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c" exitCode=0
Oct 30 11:35:36.856173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.856059    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:36.856173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.856066    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e","Type":"ContainerDied","Data":"5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c"}
Oct 30 11:35:36.856173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.856123    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e","Type":"ContainerDied","Data":"6d19abdf7d992dbbe1e0e7ac94b87450d2dc1cb76c2d088e32d5b285be4a0d93"}
Oct 30 11:35:36.856173 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.856147    4849 scope.go:122] "RemoveContainer" containerID="5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c"
Oct 30 11:35:36.858715 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.858693    4849 generic.go:356] "Generic (PLEG): container finished" podID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerID="f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332" exitCode=0
Oct 30 11:35:36.858818 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.858767    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerDied","Data":"f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332"}
Oct 30 11:35:36.861483 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.861465    4849 generic.go:356] "Generic (PLEG): container finished" podID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerID="c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da" exitCode=0
Oct 30 11:35:36.861483 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.861480    4849 generic.go:356] "Generic (PLEG): container finished" podID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerID="7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54" exitCode=0
Oct 30 11:35:36.861632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.861485    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerDied","Data":"c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da"}
Oct 30 11:35:36.861632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.861548    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerDied","Data":"7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54"}
Oct 30 11:35:36.865071 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.865050    4849 scope.go:122] "RemoveContainer" containerID="5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c"
Oct 30 11:35:36.865319 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:36.865298    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c\": container with ID starting with 5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c not found: ID does not exist" containerID="5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c"
Oct 30 11:35:36.865383 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.865329    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c"} err="failed to get container status \"5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c\": rpc error: code = NotFound desc = could not find container \"5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c\": container with ID starting with 5053ba55e394abcc2ce339e7ec6d447bef74b3f926cdd3fbd3c1dc3075be522c not found: ID does not exist"
Oct 30 11:35:36.911574 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.911553    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:36.915388 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:36.915366    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:37.386184 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.386161    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:37.439376 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439352    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-thanos-prometheus-http-client-file\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439396    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-secret-prometheus-web-tls-ca\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439434    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439459    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439483    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439520 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439510    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-tls-assets\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-tls-assets\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439545    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439600    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439671    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config-out\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.439789 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439724    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.440014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439922    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-0" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:35:37.440014 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.439955    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.440141 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.440015    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.440141 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.440073    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-kube-api-access-qcsch\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-kube-api-access-qcsch\") pod \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\" (UID: \"bc491ab1-295c-4f88-b744-80ca1ac1dedd\") "
Oct 30 11:35:37.440441 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.440413    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.441292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.441228    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-2" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:35:37.441292 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.441227    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-1" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:35:37.442531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.442510    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.444132 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.444087    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.445320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.445227    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-secret-prometheus-web-tls-ca" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.445320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.445230    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config-out" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:35:37.445320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.445251    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.445320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.445286    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-tls-assets" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:35:37.445320 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.445317    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-thanos-prometheus-http-client-file" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.447465 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.447410    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-kube-api-access-qcsch" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "kube-api-access-qcsch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:35:37.448004 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.447980    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:37.448236 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.448001    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:35:37.462577 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.462548    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config" pod "bc491ab1-295c-4f88-b744-80ca1ac1dedd" (UID: "bc491ab1-295c-4f88-b744-80ca1ac1dedd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.551871 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.551838    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-volume\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-volume\") pod \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") "
Oct 30 11:35:37.552041 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.551927    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-tls-assets\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-tls-assets\") pod \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") "
Oct 30 11:35:37.552041 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552011    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-web-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-web-config\") pod \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") "
Oct 30 11:35:37.552144 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552063    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-kube-api-access-92sww\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-kube-api-access-92sww\") pod \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") "
Oct 30 11:35:37.552144 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552127    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-alertmanager-data-science-monitoringstack-db\") pod \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") "
Oct 30 11:35:37.552265 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552160    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-cluster-tls-config\") pod \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") "
Oct 30 11:35:37.552265 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552213    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-out\") pod \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\" (UID: \"c0ce63f8-5e15-479a-912a-c2fee9cbaa10\") "
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552445    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552470    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552485    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552499    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552527    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552543    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bc491ab1-295c-4f88-b744-80ca1ac1dedd-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552558    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcsch\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-kube-api-access-qcsch\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552572    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552585    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552600    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552616    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/bc491ab1-295c-4f88-b744-80ca1ac1dedd-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552631    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bc491ab1-295c-4f88-b744-80ca1ac1dedd-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.552909 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.552677    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-alertmanager-data-science-monitoringstack-db" pod "c0ce63f8-5e15-479a-912a-c2fee9cbaa10" (UID: "c0ce63f8-5e15-479a-912a-c2fee9cbaa10"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:35:37.554925 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.554896    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-kube-api-access-92sww" pod "c0ce63f8-5e15-479a-912a-c2fee9cbaa10" (UID: "c0ce63f8-5e15-479a-912a-c2fee9cbaa10"). InnerVolumeSpecName "kube-api-access-92sww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:35:37.555170 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.555144    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-volume" pod "c0ce63f8-5e15-479a-912a-c2fee9cbaa10" (UID: "c0ce63f8-5e15-479a-912a-c2fee9cbaa10"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.555358 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.555334    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-tls-assets" pod "c0ce63f8-5e15-479a-912a-c2fee9cbaa10" (UID: "c0ce63f8-5e15-479a-912a-c2fee9cbaa10"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:35:37.556000 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.555978    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-out" pod "c0ce63f8-5e15-479a-912a-c2fee9cbaa10" (UID: "c0ce63f8-5e15-479a-912a-c2fee9cbaa10"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:35:37.559501 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.559469    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-cluster-tls-config" pod "c0ce63f8-5e15-479a-912a-c2fee9cbaa10" (UID: "c0ce63f8-5e15-479a-912a-c2fee9cbaa10"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.565112 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.565096    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:35:37.566951 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.566931    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-web-config" pod "c0ce63f8-5e15-479a-912a-c2fee9cbaa10" (UID: "c0ce63f8-5e15-479a-912a-c2fee9cbaa10"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:35:37.653456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653425    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.653456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653455    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.653580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653466    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.653580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653474    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.653580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653482    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.653580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653492    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\""
Oct 30 11:35:37.653580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653501    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.653580 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.653510    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92sww\" (UniqueName: \"kubernetes.io/projected/c0ce63f8-5e15-479a-912a-c2fee9cbaa10-kube-api-access-92sww\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:35:37.869163 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.869132    4849 generic.go:356] "Generic (PLEG): container finished" podID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerID="d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225" exitCode=0
Oct 30 11:35:37.869279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.869213    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"
Oct 30 11:35:37.869279 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.869216    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerDied","Data":"d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225"}
Oct 30 11:35:37.869360 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.869276    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1" event={"ID":"c0ce63f8-5e15-479a-912a-c2fee9cbaa10","Type":"ContainerDied","Data":"9e3af703e7dcdbdfb944a192a7dba5795622ef5675640974a69e59e32d5d179d"}
Oct 30 11:35:37.869360 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.869309    4849 scope.go:122] "RemoveContainer" containerID="f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332"
Oct 30 11:35:37.871925 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.871899    4849 generic.go:356] "Generic (PLEG): container finished" podID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerID="754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d" exitCode=0
Oct 30 11:35:37.872045 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.871958    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:37.872045 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.871976    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerDied","Data":"754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d"}
Oct 30 11:35:37.872187 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.872050    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"bc491ab1-295c-4f88-b744-80ca1ac1dedd","Type":"ContainerDied","Data":"d7f183e5f11628a418a5ed94c91f19fbc86f98a03d1eb399da8083f0785050e6"}
Oct 30 11:35:37.879524 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.879507    4849 scope.go:122] "RemoveContainer" containerID="d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225"
Oct 30 11:35:37.887867 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.887851    4849 scope.go:122] "RemoveContainer" containerID="bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00"
Oct 30 11:35:37.895417 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.895401    4849 scope.go:122] "RemoveContainer" containerID="f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332"
Oct 30 11:35:37.895679 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:37.895662    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332\": container with ID starting with f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332 not found: ID does not exist" containerID="f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332"
Oct 30 11:35:37.895746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.895690    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332"} err="failed to get container status \"f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332\": rpc error: code = NotFound desc = could not find container \"f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332\": container with ID starting with f78ebca00d61f1604ec14f474329ec80e9ab64e5f71c063edc7eced3e31f6332 not found: ID does not exist"
Oct 30 11:35:37.895746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.895711    4849 scope.go:122] "RemoveContainer" containerID="d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225"
Oct 30 11:35:37.895933 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:37.895915    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225\": container with ID starting with d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225 not found: ID does not exist" containerID="d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225"
Oct 30 11:35:37.896001 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.895942    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225"} err="failed to get container status \"d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225\": rpc error: code = NotFound desc = could not find container \"d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225\": container with ID starting with d6f6933e239b8c8c594b4caa484f1480cfaa0f5c10bcf9f44d74bf5111ba3225 not found: ID does not exist"
Oct 30 11:35:37.896001 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.895962    4849 scope.go:122] "RemoveContainer" containerID="bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00"
Oct 30 11:35:37.896230 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:37.896213    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00\": container with ID starting with bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00 not found: ID does not exist" containerID="bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00"
Oct 30 11:35:37.896297 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.896236    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00"} err="failed to get container status \"bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00\": rpc error: code = NotFound desc = could not find container \"bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00\": container with ID starting with bfca55756e19523a7dd65a7be0365c017e6db3a45d0fd32ab073e0ef0813df00 not found: ID does not exist"
Oct 30 11:35:37.896297 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.896254    4849 scope.go:122] "RemoveContainer" containerID="c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da"
Oct 30 11:35:37.903459 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.903440    4849 scope.go:122] "RemoveContainer" containerID="754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d"
Oct 30 11:35:37.911080 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.911064    4849 scope.go:122] "RemoveContainer" containerID="7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54"
Oct 30 11:35:37.918426 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.918410    4849 scope.go:122] "RemoveContainer" containerID="c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b"
Oct 30 11:35:37.928576 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.928528    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:37.929668 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.929605    4849 scope.go:122] "RemoveContainer" containerID="c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da"
Oct 30 11:35:37.930219 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:37.930199    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da\": container with ID starting with c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da not found: ID does not exist" containerID="c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da"
Oct 30 11:35:37.930295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.930226    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da"} err="failed to get container status \"c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da\": rpc error: code = NotFound desc = could not find container \"c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da\": container with ID starting with c3499452a3e91759de6cbfdb18792efe658062f3a6a21f6e248824bb90ac16da not found: ID does not exist"
Oct 30 11:35:37.930295 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.930243    4849 scope.go:122] "RemoveContainer" containerID="754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d"
Oct 30 11:35:37.930725 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:37.930707    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d\": container with ID starting with 754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d not found: ID does not exist" containerID="754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d"
Oct 30 11:35:37.930806 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.930730    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d"} err="failed to get container status \"754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d\": rpc error: code = NotFound desc = could not find container \"754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d\": container with ID starting with 754e028277c35181cc4f43c9586e16993fbf74fb065bc90679d90b8e0a001c0d not found: ID does not exist"
Oct 30 11:35:37.930806 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.930749    4849 scope.go:122] "RemoveContainer" containerID="7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54"
Oct 30 11:35:37.931059 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:37.931015    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54\": container with ID starting with 7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54 not found: ID does not exist" containerID="7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54"
Oct 30 11:35:37.931151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.931063    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54"} err="failed to get container status \"7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54\": rpc error: code = NotFound desc = could not find container \"7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54\": container with ID starting with 7436ff07444f3643d4797ef7d04015c2a62738c264e9c11590cb355e5cffbd54 not found: ID does not exist"
Oct 30 11:35:37.931151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.931079    4849 scope.go:122] "RemoveContainer" containerID="c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b"
Oct 30 11:35:37.931348 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:37.931332    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b\": container with ID starting with c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b not found: ID does not exist" containerID="c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b"
Oct 30 11:35:37.931401 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.931356    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b"} err="failed to get container status \"c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b\": rpc error: code = NotFound desc = could not find container \"c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b\": container with ID starting with c50ee214129b5055dd22313913020e3319e8956b0613de68e8558e620089358b not found: ID does not exist"
Oct 30 11:35:37.941438 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.941414    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:37.969688 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.969668    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:35:37.976499 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:37.976479    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-1"]
Oct 30 11:35:38.752897 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:38.752864    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" path="/var/lib/kubelet/pods/8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e/volumes"
Oct 30 11:35:38.753346 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:38.753331    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" path="/var/lib/kubelet/pods/bc491ab1-295c-4f88-b744-80ca1ac1dedd/volumes"
Oct 30 11:35:38.754155 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:38.754138    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" path="/var/lib/kubelet/pods/c0ce63f8-5e15-479a-912a-c2fee9cbaa10/volumes"
Oct 30 11:35:39.430682 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:35:39.430650    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:35:46.195059 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195011    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:46.195526 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195506    4849 cpu_manager.go:414] "containerMap: removing container" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="init-config-reloader"
Oct 30 11:35:46.195564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195531    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="init-config-reloader"
Oct 30 11:35:46.195564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195544    4849 cpu_manager.go:414] "containerMap: removing container" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="config-reloader"
Oct 30 11:35:46.195564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195553    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="config-reloader"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195576    4849 cpu_manager.go:414] "containerMap: removing container" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="thanos-sidecar"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195584    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="thanos-sidecar"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195597    4849 cpu_manager.go:414] "containerMap: removing container" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="prometheus"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195605    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="prometheus"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195620    4849 cpu_manager.go:414] "containerMap: removing container" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="config-reloader"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195640    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="config-reloader"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195655    4849 cpu_manager.go:414] "containerMap: removing container" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="alertmanager"
Oct 30 11:35:46.195670 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195663    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="alertmanager"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195686    4849 cpu_manager.go:414] "containerMap: removing container" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="init-config-reloader"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195696    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="init-config-reloader"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195705    4849 cpu_manager.go:414] "containerMap: removing container" podUID="8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" containerName="otc-container"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195712    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" containerName="otc-container"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195825    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="config-reloader"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195841    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="thanos-sidecar"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195851    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="prometheus"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195861    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc491ab1-295c-4f88-b744-80ca1ac1dedd" containerName="config-reloader"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195869    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0ce63f8-5e15-479a-912a-c2fee9cbaa10" containerName="alertmanager"
Oct 30 11:35:46.195913 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.195881    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="8916c0f0-5e82-418d-ad04-5e4ab5a3ae5e" containerName="otc-container"
Oct 30 11:35:46.200976 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.200949    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.220805 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.220775    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 11:35:46.220935 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.220891    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-e4cadd38\""
Oct 30 11:35:46.234896 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.234863    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 11:35:46.235030 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.234897    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 11:35:46.235030 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.234941    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 11:35:46.236932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.236908    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:46.269806 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.269777    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 11:35:46.314033 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.314002    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c075b566-73e8-462d-aa54-1d913f0fe2cf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.314147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.314099    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.314147 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.314137    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.314227 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.314167    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.314227 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.314187    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6srg\" (UniqueName: \"kubernetes.io/projected/c075b566-73e8-462d-aa54-1d913f0fe2cf-kube-api-access-n6srg\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.415339 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.415312    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c075b566-73e8-462d-aa54-1d913f0fe2cf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.415339 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.415341    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.415531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.415363    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.415531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.415426    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.415531 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.415476    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6srg\" (UniqueName: \"kubernetes.io/projected/c075b566-73e8-462d-aa54-1d913f0fe2cf-kube-api-access-n6srg\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.416146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.416123    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c075b566-73e8-462d-aa54-1d913f0fe2cf-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.417918 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.417893    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.418008 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.417937    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.418008 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.417968    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.440061 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.440039    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6srg\" (UniqueName: \"kubernetes.io/projected/c075b566-73e8-462d-aa54-1d913f0fe2cf-kube-api-access-n6srg\") pod \"data-science-collector-collector-0\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.515401 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.515350    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:35:46.820103 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.820077    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:35:46.822128 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:35:46.822098    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc075b566_73e8_462d_aa54_1d913f0fe2cf.slice/crio-0e518aa2a3f4bd4865fd4da2667e4982be40ed27563a448a504f8042c9188a39 WatchSource:0}: Error finding container 0e518aa2a3f4bd4865fd4da2667e4982be40ed27563a448a504f8042c9188a39: Status 404 returned error can't find the container with id 0e518aa2a3f4bd4865fd4da2667e4982be40ed27563a448a504f8042c9188a39
Oct 30 11:35:46.829771 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.824703    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:35:46.829771 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.824832    4849 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 11:35:46.922480 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.922434    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c075b566-73e8-462d-aa54-1d913f0fe2cf","Type":"ContainerStarted","Data":"6d6780b41cafc1e8087b6bd930aa6ab0a10e02334a4cb3af31af896b16ff2437"}
Oct 30 11:35:46.922660 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.922535    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c075b566-73e8-462d-aa54-1d913f0fe2cf","Type":"ContainerStarted","Data":"0e518aa2a3f4bd4865fd4da2667e4982be40ed27563a448a504f8042c9188a39"}
Oct 30 11:35:46.976317 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:46.976198    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=0.976180774 podStartE2EDuration="976.180774ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:35:46.973909271 +0000 UTC m=+3044.354358710" watchObservedRunningTime="2026-10-30 11:35:46.976180774 +0000 UTC m=+3044.356630214"
Oct 30 11:35:49.665541 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.665510    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:35:49.669036 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.669005    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.685756 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.685733    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-web-config\""
Oct 30 11:35:49.685932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.685897    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:35:49.685932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.685909    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-generated\""
Oct 30 11:35:49.686149 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.686130    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"alertmanager-data-science-monitoringstack-cluster-tls-config\""
Oct 30 11:35:49.699406 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.699387    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-alertmanager-dockercfg-xzf25\""
Oct 30 11:35:49.708444 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.708423    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:35:49.741202 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.741169    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.741202 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.741197    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.741348 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.741219    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ktms\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-kube-api-access-7ktms\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.741387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.741347    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.741423 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.741390    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.741496 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.741480    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.741529 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.741511    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.794894 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.794863    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:49.798695 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.798681    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.824639 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.824612    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-1\""
Oct 30 11:35:49.824799 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.824780    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-2\""
Oct 30 11:35:49.825004 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.824990    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-rulefiles-0\""
Oct 30 11:35:49.834053 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.832955    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:49.842209 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842158    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.842209 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842194    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.842374 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842223    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ktms\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-kube-api-access-7ktms\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.842374 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842258    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.842374 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842304    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.842374 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842361    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.842594 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842405    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.842836 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.842807    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-alertmanager-data-science-monitoringstack-db\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.844802 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.844774    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-tls-assets\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.845042 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.844996    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-out\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.845113 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.845098    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-volume\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.845222 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.845202    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-cluster-tls-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.845577 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.845561    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-web-config\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.873892 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.873864    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ktms\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-kube-api-access-7ktms\") pod \"alertmanager-data-science-monitoringstack-0\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") " pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:49.885846 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.885815    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-web-config\""
Oct 30 11:35:49.885970 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.885879    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-monitoringstack-prometheus-dockercfg-m56vk\""
Oct 30 11:35:49.885970 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.885891    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-web-tls-ca\""
Oct 30 11:35:49.886172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.885968    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-thanos-prometheus-http-client-file\""
Oct 30 11:35:49.886172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.886083    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-operated-tls\""
Oct 30 11:35:49.886172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.886094    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack-tls-assets-0\""
Oct 30 11:35:49.886172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.886094    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"prometheus-data-science-monitoringstack\""
Oct 30 11:35:49.943139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943117    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943236 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943154    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943236 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943185    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943312 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943277    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943312 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943305    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943334    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943495 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943476    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943573 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943503    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqqn2\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-kube-api-access-xqqn2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943573 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943562    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943596    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943627    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.943740 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.943720    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:49.987201 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:49.987181    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:50.044815 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.044775    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.044911 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.044844    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.044911 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.044878    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.044995 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.044922    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.044995 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.044956    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.044995 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.044980    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqqn2\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-kube-api-access-xqqn2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.045114 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.045042    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.045114 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.045071    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.045181 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.045140    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.045215 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.045177    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.045247 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.045225    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.045294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.045263    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.047244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.046886    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.047633 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.047603    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config-out\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.047718 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.047636    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.048399 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.048374    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.049930 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.049878    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.050206 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.050164    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.050784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.050764    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-secret-prometheus-web-tls-ca\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.051930 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.051906    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.052052 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.051996    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-thanos-prometheus-http-client-file\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.055699 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.055672    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-tls-assets\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.057638 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.057615    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.073195 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.073172    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqqn2\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-kube-api-access-xqqn2\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:50.308856 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.308835    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:35:50.310408 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:35:50.310384    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1ea2d0e_6cbc_4240_afcd_6e65e68ab69c.slice/crio-bbe6fc0811a59e71a18b70d5edcc4c2596a9b05e8bd02e301d0a2e768f251226 WatchSource:0}: Error finding container bbe6fc0811a59e71a18b70d5edcc4c2596a9b05e8bd02e301d0a2e768f251226: Status 404 returned error can't find the container with id bbe6fc0811a59e71a18b70d5edcc4c2596a9b05e8bd02e301d0a2e768f251226
Oct 30 11:35:50.954115 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.954046    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerStarted","Data":"dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9"}
Oct 30 11:35:50.954491 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:50.954127    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerStarted","Data":"bbe6fc0811a59e71a18b70d5edcc4c2596a9b05e8bd02e301d0a2e768f251226"}
Oct 30 11:35:54.975869 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:54.975831    4849 generic.go:356] "Generic (PLEG): container finished" podID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerID="dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9" exitCode=0
Oct 30 11:35:54.976258 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:54.975901    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerDied","Data":"dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9"}
Oct 30 11:35:55.982419 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:55.982392    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerStarted","Data":"a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606"}
Oct 30 11:35:55.982830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:55.982448    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerStarted","Data":"8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67"}
Oct 30 11:35:55.982830 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:55.982618    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:35:56.048140 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:56.048094    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podStartSLOduration=3.5624603500000003 podStartE2EDuration="7.048083307s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.485622957s" isStatefulPod=false podCreationTimestamp="2026-10-30 11:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:35:56.032702284 +0000 UTC m=+3053.413151725" watchObservedRunningTime="2026-10-30 11:35:56.048083307 +0000 UTC m=+3053.428532754"
Oct 30 11:35:57.404720 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.404685    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:57.437492 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.437460    4849 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") device path: \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:57.505984 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.505949    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:57.506146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.506089    4849 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") DevicePath \"\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:57.522364 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.522336    4849 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:57.625342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.625320    4849 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/5f4e377c3e259c071aab429e31ca61846b879cc3476e2f02b70e888ccdcdb590/globalmount\"" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:57.653823 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.653802    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"prometheus-data-science-monitoringstack-0\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") " pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:57.918285 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:57.918256    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:35:58.254057 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:58.254015    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:35:58.255403 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:35:58.255371    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee81b59_89ec_4e2b_8fdb_8a547f134d5f.slice/crio-e1c9028c497942405d969e6d7dfe1cacf79437f8124629e11004aea6cd6acf60 WatchSource:0}: Error finding container e1c9028c497942405d969e6d7dfe1cacf79437f8124629e11004aea6cd6acf60: Status 404 returned error can't find the container with id e1c9028c497942405d969e6d7dfe1cacf79437f8124629e11004aea6cd6acf60
Oct 30 11:35:58.999456 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:58.999390    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerStarted","Data":"7ad1da18ec221411aef7c511dedf5138725314c2246afa0c8df7d6e506f74fda"}
Oct 30 11:35:58.999777 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:35:58.999465    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerStarted","Data":"e1c9028c497942405d969e6d7dfe1cacf79437f8124629e11004aea6cd6acf60"}
Oct 30 11:36:01.989172 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:01.989145    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:36:02.021996 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:02.021972    4849 generic.go:356] "Generic (PLEG): container finished" podID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerID="7ad1da18ec221411aef7c511dedf5138725314c2246afa0c8df7d6e506f74fda" exitCode=0
Oct 30 11:36:02.022131 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:02.022050    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerDied","Data":"7ad1da18ec221411aef7c511dedf5138725314c2246afa0c8df7d6e506f74fda"}
Oct 30 11:36:03.029354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:03.029286    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerStarted","Data":"a6fe25154afc4357a8609626cdce3a75334a9656dfd3367438a18811c7bb4497"}
Oct 30 11:36:03.029354 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:03.029352    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerStarted","Data":"859016ff86018f2a0f6979326281688ebbe4a81eaea99bab1bfcda3d2fa5c422"}
Oct 30 11:36:03.029735 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:03.029378    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerStarted","Data":"408bd44574fbd90585c1b8f72004812a82493c470652a8e19f9f443dd9ae5be0"}
Oct 30 11:36:03.127301 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:03.127251    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podStartSLOduration=10.840524969 podStartE2EDuration="14.127238184s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.286713215s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:36:03.111397568 +0000 UTC m=+3060.491847006" watchObservedRunningTime="2026-10-30 11:36:03.127238184 +0000 UTC m=+3060.507687622"
Oct 30 11:36:07.919189 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:07.919151    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:36:09.432809 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:36:09.432774    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:36:12.919193 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:12.919157    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:36:12.924745 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:12.924715    4849 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:36:13.084778 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:13.084755    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:36:16.320972 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.320938    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:36:16.321405 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.321197    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="alertmanager" containerID="cri-o://8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67" gracePeriod=120
Oct 30 11:36:16.321405 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.321241    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="config-reloader" containerID="cri-o://a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606" gracePeriod=120
Oct 30 11:36:16.340641 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.340619    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:36:16.340917 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.340886    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="prometheus" containerID="cri-o://408bd44574fbd90585c1b8f72004812a82493c470652a8e19f9f443dd9ae5be0" gracePeriod=600
Oct 30 11:36:16.341050 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.340892    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="thanos-sidecar" containerID="cri-o://a6fe25154afc4357a8609626cdce3a75334a9656dfd3367438a18811c7bb4497" gracePeriod=600
Oct 30 11:36:16.341050 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.340889    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="config-reloader" containerID="cri-o://859016ff86018f2a0f6979326281688ebbe4a81eaea99bab1bfcda3d2fa5c422" gracePeriod=600
Oct 30 11:36:16.970063 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.970010    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:36:16.970244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.970223    4849 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="c075b566-73e8-462d-aa54-1d913f0fe2cf" containerName="otc-container" containerID="cri-o://6d6780b41cafc1e8087b6bd930aa6ab0a10e02334a4cb3af31af896b16ff2437" gracePeriod=30
Oct 30 11:36:16.988377 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:16.988317    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.130:9093/-/ready\": dial tcp 10.132.0.130:9093: connect: connection refused"
Oct 30 11:36:17.103977 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.103945    4849 generic.go:356] "Generic (PLEG): container finished" podID="c075b566-73e8-462d-aa54-1d913f0fe2cf" containerID="6d6780b41cafc1e8087b6bd930aa6ab0a10e02334a4cb3af31af896b16ff2437" exitCode=0
Oct 30 11:36:17.104146 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.104047    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c075b566-73e8-462d-aa54-1d913f0fe2cf","Type":"ContainerDied","Data":"6d6780b41cafc1e8087b6bd930aa6ab0a10e02334a4cb3af31af896b16ff2437"}
Oct 30 11:36:17.108429 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.108405    4849 generic.go:356] "Generic (PLEG): container finished" podID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerID="a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606" exitCode=0
Oct 30 11:36:17.108564 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.108490    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerDied","Data":"a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606"}
Oct 30 11:36:17.111578 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.111550    4849 generic.go:356] "Generic (PLEG): container finished" podID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerID="a6fe25154afc4357a8609626cdce3a75334a9656dfd3367438a18811c7bb4497" exitCode=0
Oct 30 11:36:17.111704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.111580    4849 generic.go:356] "Generic (PLEG): container finished" podID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerID="859016ff86018f2a0f6979326281688ebbe4a81eaea99bab1bfcda3d2fa5c422" exitCode=0
Oct 30 11:36:17.111704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.111581    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerDied","Data":"a6fe25154afc4357a8609626cdce3a75334a9656dfd3367438a18811c7bb4497"}
Oct 30 11:36:17.111704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.111590    4849 generic.go:356] "Generic (PLEG): container finished" podID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerID="408bd44574fbd90585c1b8f72004812a82493c470652a8e19f9f443dd9ae5be0" exitCode=0
Oct 30 11:36:17.111704 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.111635    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerDied","Data":"859016ff86018f2a0f6979326281688ebbe4a81eaea99bab1bfcda3d2fa5c422"}
Oct 30 11:36:17.111899 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.111737    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerDied","Data":"408bd44574fbd90585c1b8f72004812a82493c470652a8e19f9f443dd9ae5be0"}
Oct 30 11:36:17.138529 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.136338    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:36:17.261334 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261278    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-secret-prometheus-web-tls-ca\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261472 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261450    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-tls-assets\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-tls-assets\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261556 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261497    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261556 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261529    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-kube-api-access-xqqn2\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-kube-api-access-xqqn2\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261589    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261679 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261646    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config-out\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261795 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261680    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-0\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261795 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261729    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261795 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261768    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-thanos-prometheus-http-client-file\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261863    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261895    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.261956 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261925    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-2\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.262170 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.261960    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-1\") pod \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\" (UID: \"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f\") "
Oct 30 11:36:17.263223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.262576    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-1" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:36:17.263223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.262711    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-0" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:36:17.263223 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.262899    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-2" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "prometheus-data-science-monitoringstack-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:36:17.264252 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.264223    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config-out" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:36:17.264390 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.264372    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-tls-assets" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:36:17.266366 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.266329    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-thanos-prometheus-http-client-file" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.266366 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.266356    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-kube-api-access-xqqn2" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "kube-api-access-xqqn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:36:17.266523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.266392    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-key-prometheus-operated-tls-f3457c70" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "web-config-tls-secret-key-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.266523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.266407    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-secret-prometheus-web-tls-ca" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "secret-prometheus-web-tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.266523 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.266443    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.267382 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.267361    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:36:17.267741 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.267723    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "web-config-tls-secret-cert-prometheus-operated-tls-f3457c70". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.269359 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.269332    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 11:36:17.278314 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.278292    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config" pod "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" (UID: "2ee81b59-89ec-4e2b-8fdb-8a547f134d5f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.363225 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363200    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-tls-certs\") pod \"c075b566-73e8-462d-aa54-1d913f0fe2cf\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") "
Oct 30 11:36:17.363624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363255    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-data-science-collector-ta-client-cert\") pod \"c075b566-73e8-462d-aa54-1d913f0fe2cf\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") "
Oct 30 11:36:17.363624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363292    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c075b566-73e8-462d-aa54-1d913f0fe2cf-otc-internal\" (UniqueName: \"kubernetes.io/configmap/c075b566-73e8-462d-aa54-1d913f0fe2cf-otc-internal\") pod \"c075b566-73e8-462d-aa54-1d913f0fe2cf\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") "
Oct 30 11:36:17.363624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363339    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c075b566-73e8-462d-aa54-1d913f0fe2cf-kube-api-access-n6srg\" (UniqueName: \"kubernetes.io/projected/c075b566-73e8-462d-aa54-1d913f0fe2cf-kube-api-access-n6srg\") pod \"c075b566-73e8-462d-aa54-1d913f0fe2cf\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") "
Oct 30 11:36:17.363624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363367    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-monitoring-tls-certs\") pod \"c075b566-73e8-462d-aa54-1d913f0fe2cf\" (UID: \"c075b566-73e8-462d-aa54-1d913f0fe2cf\") "
Oct 30 11:36:17.363624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363599    4849 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" "
Oct 30 11:36:17.363624 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363623    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-key-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-key-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363641    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363657    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-1\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363675    4849 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-web-tls-ca\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-secret-prometheus-web-tls-ca\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363679    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c075b566-73e8-462d-aa54-1d913f0fe2cf-otc-internal" pod "c075b566-73e8-462d-aa54-1d913f0fe2cf" (UID: "c075b566-73e8-462d-aa54-1d913f0fe2cf"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363689    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363716    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363731    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqqn2\" (UniqueName: \"kubernetes.io/projected/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-kube-api-access-xqqn2\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363744    4849 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363758    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363774    4849 reconciler_common.go:299] "Volume detached for volume \"prometheus-data-science-monitoringstack-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-prometheus-data-science-monitoringstack-rulefiles-0\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363788    4849 reconciler_common.go:299] "Volume detached for volume \"web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-web-config-tls-secret-cert-prometheus-operated-tls-f3457c70\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.363968 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.363804    4849 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.365858 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.365815    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-tls-certs" pod "c075b566-73e8-462d-aa54-1d913f0fe2cf" (UID: "c075b566-73e8-462d-aa54-1d913f0fe2cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.366002 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.365976    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-data-science-collector-ta-client-cert" pod "c075b566-73e8-462d-aa54-1d913f0fe2cf" (UID: "c075b566-73e8-462d-aa54-1d913f0fe2cf"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.366271 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.366175    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c075b566-73e8-462d-aa54-1d913f0fe2cf-kube-api-access-n6srg" pod "c075b566-73e8-462d-aa54-1d913f0fe2cf" (UID: "c075b566-73e8-462d-aa54-1d913f0fe2cf"). InnerVolumeSpecName "kube-api-access-n6srg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:36:17.366798 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.366773    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-monitoring-tls-certs" pod "c075b566-73e8-462d-aa54-1d913f0fe2cf" (UID: "c075b566-73e8-462d-aa54-1d913f0fe2cf"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:17.381294 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.381275    4849 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396") on node "ip-10-0-140-230.ec2.internal"
Oct 30 11:36:17.465145 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.465118    4849 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-data-science-collector-ta-client-cert\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.465145 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.465144    4849 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c075b566-73e8-462d-aa54-1d913f0fe2cf-otc-internal\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.465311 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.465155    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6srg\" (UniqueName: \"kubernetes.io/projected/c075b566-73e8-462d-aa54-1d913f0fe2cf-kube-api-access-n6srg\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.465311 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.465165    4849 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-monitoring-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:17.465311 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.465175    4849 reconciler_common.go:299] "Volume detached for volume \"pvc-ae2fc6cc-1a29-41df-b109-2b8489d4e098\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-02d1e006dc6512396\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"csi-3fc788cadf9563d8ef3d182bf71d2ac2362aebcf39ba983a2d3ebe3c31b6ca27\""
Oct 30 11:36:17.465311 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:17.465183    4849 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c075b566-73e8-462d-aa54-1d913f0fe2cf-tls-certs\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:18.125674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.120152    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 11:36:18.125674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.120182    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"c075b566-73e8-462d-aa54-1d913f0fe2cf","Type":"ContainerDied","Data":"0e518aa2a3f4bd4865fd4da2667e4982be40ed27563a448a504f8042c9188a39"}
Oct 30 11:36:18.125674 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.120282    4849 scope.go:122] "RemoveContainer" containerID="6d6780b41cafc1e8087b6bd930aa6ab0a10e02334a4cb3af31af896b16ff2437"
Oct 30 11:36:18.151425 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.151399    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0" event={"ID":"2ee81b59-89ec-4e2b-8fdb-8a547f134d5f","Type":"ContainerDied","Data":"e1c9028c497942405d969e6d7dfe1cacf79437f8124629e11004aea6cd6acf60"}
Oct 30 11:36:18.151617 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.151441    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"
Oct 30 11:36:18.156168 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.156149    4849 scope.go:122] "RemoveContainer" containerID="a6fe25154afc4357a8609626cdce3a75334a9656dfd3367438a18811c7bb4497"
Oct 30 11:36:18.166207 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.166187    4849 scope.go:122] "RemoveContainer" containerID="859016ff86018f2a0f6979326281688ebbe4a81eaea99bab1bfcda3d2fa5c422"
Oct 30 11:36:18.174287 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.174270    4849 scope.go:122] "RemoveContainer" containerID="408bd44574fbd90585c1b8f72004812a82493c470652a8e19f9f443dd9ae5be0"
Oct 30 11:36:18.181784 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.181766    4849 scope.go:122] "RemoveContainer" containerID="7ad1da18ec221411aef7c511dedf5138725314c2246afa0c8df7d6e506f74fda"
Oct 30 11:36:18.207391 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.207372    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:36:18.252033 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.252001    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 11:36:18.313479 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.310039    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:36:18.322057 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.322034    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/prometheus-data-science-monitoringstack-0"]
Oct 30 11:36:18.753592 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.753563    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" path="/var/lib/kubelet/pods/2ee81b59-89ec-4e2b-8fdb-8a547f134d5f/volumes"
Oct 30 11:36:18.754123 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:18.754107    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c075b566-73e8-462d-aa54-1d913f0fe2cf" path="/var/lib/kubelet/pods/c075b566-73e8-462d-aa54-1d913f0fe2cf/volumes"
Oct 30 11:36:21.989235 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:21.989194    4849 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="alertmanager" probeResult="failure" output="Get \"http://10.132.0.130:9093/-/ready\": dial tcp 10.132.0.130:9093: connect: connection refused"
Oct 30 11:36:26.577839 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.577811    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:36:26.738311 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738243    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-tls-assets\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-tls-assets\") pod \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") "
Oct 30 11:36:26.738311 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738289    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-out\") pod \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") "
Oct 30 11:36:26.738558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738332    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-kube-api-access-7ktms\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-kube-api-access-7ktms\") pod \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") "
Oct 30 11:36:26.738558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738436    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-volume\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-volume\") pod \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") "
Oct 30 11:36:26.738558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738471    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-web-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-web-config\") pod \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") "
Oct 30 11:36:26.738558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738517    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-alertmanager-data-science-monitoringstack-db\") pod \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") "
Oct 30 11:36:26.738558 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738552    4849 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-cluster-tls-config\") pod \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\" (UID: \"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c\") "
Oct 30 11:36:26.738932 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.738904    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-alertmanager-data-science-monitoringstack-db" pod "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" (UID: "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c"). InnerVolumeSpecName "alertmanager-data-science-monitoringstack-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:36:26.741066 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.741003    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-volume" pod "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" (UID: "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:26.741235 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.741073    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-out" pod "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" (UID: "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 11:36:26.741235 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.741142    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-tls-assets" pod "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" (UID: "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:36:26.741365 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.741270    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-kube-api-access-7ktms" pod "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" (UID: "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c"). InnerVolumeSpecName "kube-api-access-7ktms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 11:36:26.745114 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.745088    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-cluster-tls-config" pod "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" (UID: "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:26.752170 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.752144    4849 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-web-config" pod "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" (UID: "d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 11:36:26.839120 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.839096    4849 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-out\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:26.839120 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.839118    4849 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ktms\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-kube-api-access-7ktms\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:26.839260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.839132    4849 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-config-volume\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:26.839260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.839144    4849 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-web-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:26.839260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.839155    4849 reconciler_common.go:299] "Volume detached for volume \"alertmanager-data-science-monitoringstack-db\" (UniqueName: \"kubernetes.io/empty-dir/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-alertmanager-data-science-monitoringstack-db\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:26.839260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.839169    4849 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-cluster-tls-config\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:26.839260 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:26.839182    4849 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c-tls-assets\") on node \"ip-10-0-140-230.ec2.internal\" DevicePath \"\""
Oct 30 11:36:27.203797 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.203771    4849 generic.go:356] "Generic (PLEG): container finished" podID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerID="8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67" exitCode=0
Oct 30 11:36:27.203903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.203829    4849 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"
Oct 30 11:36:27.203903 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.203853    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerDied","Data":"8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67"}
Oct 30 11:36:27.203980 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.203916    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0" event={"ID":"d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c","Type":"ContainerDied","Data":"bbe6fc0811a59e71a18b70d5edcc4c2596a9b05e8bd02e301d0a2e768f251226"}
Oct 30 11:36:27.203980 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.203944    4849 scope.go:122] "RemoveContainer" containerID="a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606"
Oct 30 11:36:27.212545 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.212527    4849 scope.go:122] "RemoveContainer" containerID="8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67"
Oct 30 11:36:27.220557 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.220538    4849 scope.go:122] "RemoveContainer" containerID="dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9"
Oct 30 11:36:27.228011 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.227985    4849 scope.go:122] "RemoveContainer" containerID="a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606"
Oct 30 11:36:27.228272 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:36:27.228251    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606\": container with ID starting with a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606 not found: ID does not exist" containerID="a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606"
Oct 30 11:36:27.228355 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.228285    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606"} err="failed to get container status \"a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606\": rpc error: code = NotFound desc = could not find container \"a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606\": container with ID starting with a49091e43abb601eeb31bb58daa433aa09d223def391f7fdbb79bd80ae4b9606 not found: ID does not exist"
Oct 30 11:36:27.228355 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.228309    4849 scope.go:122] "RemoveContainer" containerID="8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67"
Oct 30 11:36:27.228591 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:36:27.228574    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67\": container with ID starting with 8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67 not found: ID does not exist" containerID="8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67"
Oct 30 11:36:27.228632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.228599    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67"} err="failed to get container status \"8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67\": rpc error: code = NotFound desc = could not find container \"8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67\": container with ID starting with 8c1e831668fa4c524b699246f9f9d763876285f0717179375e2c5b0f67221b67 not found: ID does not exist"
Oct 30 11:36:27.228632 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.228615    4849 scope.go:122] "RemoveContainer" containerID="dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9"
Oct 30 11:36:27.228847 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:36:27.228829    4849 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9\": container with ID starting with dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9 not found: ID does not exist" containerID="dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9"
Oct 30 11:36:27.228912 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.228854    4849 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9"} err="failed to get container status \"dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9\": rpc error: code = NotFound desc = could not find container \"dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9\": container with ID starting with dffee4bb7564acdfe61eb6ecd67f425ee3ff784aa81666e8ac2d31423ec4dff9 not found: ID does not exist"
Oct 30 11:36:27.275841 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.275811    4849 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:36:27.289717 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:27.289695    4849 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/alertmanager-data-science-monitoringstack-0"]
Oct 30 11:36:28.752880 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:28.752840    4849 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" path="/var/lib/kubelet/pods/d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c/volumes"
Oct 30 11:36:36.599995 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:36.599959    4849 ???:1] "http: TLS handshake error from 10.0.140.230:58208: EOF"
Oct 30 11:36:36.611779 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:36.611750    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lkmz6_bc612ba8-81c6-4d10-aecf-f8efdc6fea4f/global-pull-secret-syncer/0.log"
Oct 30 11:36:37.006139 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:37.006070    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-8pgqv_f384be3a-0f13-4b0a-ba1b-d63d2095b9d2/connection-checker/0.log"
Oct 30 11:36:37.127005 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:37.126979    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-r9ns7_e3cc17ea-0cec-4a71-882d-2df0457dd957/connection-checker/0.log"
Oct 30 11:36:37.250781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:37.250756    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-xfp2x_418cb483-75bb-49b0-9c67-f3b70c26c2b4/connection-checker/0.log"
Oct 30 11:36:37.754251 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:37.754226    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q6ddt_8bfa87f2-a44f-4a2c-bbb0-f1a6668e1657/konnectivity-agent/0.log"
Oct 30 11:36:38.160878 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:38.160849    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-230.ec2.internal_d42823a77ed791ee517181c1c853da97/haproxy/0.log"
Oct 30 11:36:39.430459 ip-10-0-140-230 kubenswrapper[4849]: E1030 11:36:39.430425    4849 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2kxlm" containerName="extractor"
Oct 30 11:36:43.857671 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:43.857633    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-lw745_1cbcd9b8-ce20-494c-9a66-475272f2a561/cluster-monitoring-operator/0.log"
Oct 30 11:36:45.022484 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.022406    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dj2v9_2686d713-b3ff-4f5a-8183-adff6efb48d0/node-exporter/0.log"
Oct 30 11:36:45.154591 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.154562    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dj2v9_2686d713-b3ff-4f5a-8183-adff6efb48d0/kube-rbac-proxy/0.log"
Oct 30 11:36:45.154746 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.154689    4849 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"]
Oct 30 11:36:45.155003 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.154991    4849 cpu_manager.go:414] "containerMap: removing container" podUID="c075b566-73e8-462d-aa54-1d913f0fe2cf" containerName="otc-container"
Oct 30 11:36:45.155062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155004    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c075b566-73e8-462d-aa54-1d913f0fe2cf" containerName="otc-container"
Oct 30 11:36:45.155062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155015    4849 cpu_manager.go:414] "containerMap: removing container" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="thanos-sidecar"
Oct 30 11:36:45.155062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155033    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="thanos-sidecar"
Oct 30 11:36:45.155062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155043    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="alertmanager"
Oct 30 11:36:45.155062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155049    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="alertmanager"
Oct 30 11:36:45.155062 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155060    4849 cpu_manager.go:414] "containerMap: removing container" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="init-config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155065    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="init-config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155074    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155079    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155087    4849 cpu_manager.go:414] "containerMap: removing container" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155092    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155106    4849 cpu_manager.go:414] "containerMap: removing container" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="init-config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155110    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="init-config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155116    4849 cpu_manager.go:414] "containerMap: removing container" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="prometheus"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155121    4849 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="prometheus"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155180    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155188    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="config-reloader"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155196    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="prometheus"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155204    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="c075b566-73e8-462d-aa54-1d913f0fe2cf" containerName="otc-container"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155209    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ee81b59-89ec-4e2b-8fdb-8a547f134d5f" containerName="thanos-sidecar"
Oct 30 11:36:45.155244 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.155215    4849 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1ea2d0e-6cbc-4240-afcd-6e65e68ab69c" containerName="alertmanager"
Oct 30 11:36:45.158333 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.158312    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.170682 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.170661    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-proc\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.170781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.170686    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-podres\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.170781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.170711    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-lib-modules\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.170781 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.170730    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-sys\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.170887 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.170851    4849 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4krxh\" (UniqueName: \"kubernetes.io/projected/cde009a1-3de3-4b91-9930-c7e2f992d4e7-kube-api-access-4krxh\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.179633 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.179612    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"]
Oct 30 11:36:45.233123 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.233094    4849 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tgn5g\"/\"default-dockercfg-qskz5\""
Oct 30 11:36:45.233210 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.233125    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tgn5g\"/\"kube-root-ca.crt\""
Oct 30 11:36:45.233210 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.233125    4849 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tgn5g\"/\"openshift-service-ca.crt\""
Oct 30 11:36:45.271151 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271127    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-sys\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271181    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4krxh\" (UniqueName: \"kubernetes.io/projected/cde009a1-3de3-4b91-9930-c7e2f992d4e7-kube-api-access-4krxh\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271237 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271196    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-proc\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271245    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-proc\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271248    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-sys\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271342 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271301    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-podres\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271363    4849 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-lib-modules\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271416    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-podres\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.271521 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.271466    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cde009a1-3de3-4b91-9930-c7e2f992d4e7-lib-modules\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.279744 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.279695    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dj2v9_2686d713-b3ff-4f5a-8183-adff6efb48d0/init-textfile/0.log"
Oct 30 11:36:45.296229 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.296210    4849 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4krxh\" (UniqueName: \"kubernetes.io/projected/cde009a1-3de3-4b91-9930-c7e2f992d4e7-kube-api-access-4krxh\") pod \"perf-node-gather-daemonset-btfjh\" (UID: \"cde009a1-3de3-4b91-9930-c7e2f992d4e7\") " pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.470948 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.470929    4849 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:45.772446 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:45.772423    4849 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"]
Oct 30 11:36:45.774709 ip-10-0-140-230 kubenswrapper[4849]: W1030 11:36:45.774683    4849 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcde009a1_3de3_4b91_9930_c7e2f992d4e7.slice/crio-b7472fa65035bde7a9d3a9e519f27cbc1f99b856763636bf50b460b0190ce3b6 WatchSource:0}: Error finding container b7472fa65035bde7a9d3a9e519f27cbc1f99b856763636bf50b460b0190ce3b6: Status 404 returned error can't find the container with id b7472fa65035bde7a9d3a9e519f27cbc1f99b856763636bf50b460b0190ce3b6
Oct 30 11:36:46.300387 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:46.300357    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh" event={"ID":"cde009a1-3de3-4b91-9930-c7e2f992d4e7","Type":"ContainerStarted","Data":"b544383ea6cb0a478f7fdb9c247aa405a82b6ffb9d7c384ea6a701f0d66f9e50"}
Oct 30 11:36:46.300722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:46.300403    4849 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh" event={"ID":"cde009a1-3de3-4b91-9930-c7e2f992d4e7","Type":"ContainerStarted","Data":"b7472fa65035bde7a9d3a9e519f27cbc1f99b856763636bf50b460b0190ce3b6"}
Oct 30 11:36:46.300722 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:46.300434    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:46.345111 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:46.345068    4849 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh" podStartSLOduration=1.345053415 podStartE2EDuration="1.345053415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 11:36:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 11:36:46.344323072 +0000 UTC m=+3103.724772510" watchObservedRunningTime="2026-10-30 11:36:46.345053415 +0000 UTC m=+3103.725502851"
Oct 30 11:36:47.497827 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:47.497792    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-696d67977-zgswj_6c28f4af-cc57-4c19-baf3-eaeaa07e7853/telemeter-client/0.log"
Oct 30 11:36:47.621082 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:47.621052    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-696d67977-zgswj_6c28f4af-cc57-4c19-baf3-eaeaa07e7853/reload/0.log"
Oct 30 11:36:47.744236 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:47.744204    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-696d67977-zgswj_6c28f4af-cc57-4c19-baf3-eaeaa07e7853/kube-rbac-proxy/0.log"
Oct 30 11:36:49.883724 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:49.883637    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-fdv8p_3138c05b-7ff7-463e-9d91-ea7fab54a541/networking-console-plugin/0.log"
Oct 30 11:36:50.887923 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:50.887898    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/2.log"
Oct 30 11:36:50.895466 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:50.895441    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2hmr5_909137f2-6cf7-4fc8-8bcd-b0620b87aa5a/console-operator/3.log"
Oct 30 11:36:52.315393 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:52.315367    4849 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tgn5g/perf-node-gather-daemonset-btfjh"
Oct 30 11:36:53.027505 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:53.027474    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-5xkvn_b8b2a817-e3d9-4361-98cd-c0a40c93b84d/volume-data-source-validator/0.log"
Oct 30 11:36:55.457675 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:55.457641    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mx9gc_2fba2151-22de-4a03-afcb-1d89b9c32a2b/dns/0.log"
Oct 30 11:36:55.580981 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:55.580951    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mx9gc_2fba2151-22de-4a03-afcb-1d89b9c32a2b/kube-rbac-proxy/0.log"
Oct 30 11:36:56.006313 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:56.006285    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-887wc_ded181d7-f3c4-4324-946f-2770e8e3d0fd/dns-node-resolver/0.log"
Oct 30 11:36:57.655186 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:57.655153    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxz95_7add3842-20f6-49eb-89ad-02078f7767ef/node-ca/0.log"
Oct 30 11:36:59.519863 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:36:59.519832    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65668b76cf-ckhgx_9bd59176-1f2c-4169-a6bc-45ca82e270eb/router/0.log"
Oct 30 11:37:00.419760 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:37:00.419729    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7kq9h_d88e5148-03db-43f8-a46f-107d8d42af0f/serve-healthcheck-canary/0.log"
Oct 30 11:37:01.626134 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:37:01.626098    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-v69fn_9e199587-19b7-4a4b-bdd0-761b7a6e475f/insights-operator/1.log"
Oct 30 11:37:01.628565 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:37:01.628543    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-v69fn_9e199587-19b7-4a4b-bdd0-761b7a6e475f/insights-operator/2.log"
Oct 30 11:37:01.732150 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:37:01.732123    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2kxlm_07ca9bba-a38c-48af-9a31-f71bad8bb876/kube-rbac-proxy/0.log"
Oct 30 11:37:01.858009 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:37:01.857984    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2kxlm_07ca9bba-a38c-48af-9a31-f71bad8bb876/exporter/0.log"
Oct 30 11:37:01.982374 ip-10-0-140-230 kubenswrapper[4849]: I1030 11:37:01.982298    4849 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2kxlm_07ca9bba-a38c-48af-9a31-f71bad8bb876/extractor/0.log"