Jul 06 23:46:12.551926 ip-10-0-130-92 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 23:46:12.551937 ip-10-0-130-92 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 23:46:12.551944 ip-10-0-130-92 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 23:46:12.552216 ip-10-0-130-92 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 23:46:23.850206 ip-10-0-130-92 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 23:46:23.850223 ip-10-0-130-92 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 69ba39c9351f4c56ad62f3cc2a6415fc --
Jul 06 23:48:18.833713 ip-10-0-130-92 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 23:48:19.163502 ip-10-0-130-92 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:48:19.163502 ip-10-0-130-92 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 23:48:19.163502 ip-10-0-130-92 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:48:19.164166 ip-10-0-130-92 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 23:48:19.164166 ip-10-0-130-92 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:48:19.164365 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.164279    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 23:48:19.167818 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167797    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:48:19.167818 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167816    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:48:19.167818 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167822    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:48:19.167818 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167826    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167831    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167835    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167840    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167844    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167848    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167851    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167856    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167859    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167863    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167868    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167871    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167875    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167880    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167883    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167887    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167896    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167900    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167904    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167907    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:48:19.168062 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167911    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167915    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167919    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167923    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167927    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167931    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167935    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167939    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167943    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167947    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167951    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167956    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167960    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167964    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167970    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167974    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167978    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167982    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167986    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167990    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:48:19.168901 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167995    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.167999    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168007    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168013    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168018    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168022    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168026    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168031    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168035    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168039    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168043    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168047    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168051    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168055    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168059    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168063    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168067    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168071    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168075    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:48:19.169755 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168079    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168083    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168087    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168091    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168095    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168099    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168104    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168108    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168112    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168118    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168122    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168129    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168133    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168137    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168142    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168147    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168151    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168155    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168161    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:48:19.170217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168166    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168171    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168175    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168179    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168183    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168187    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168191    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168195    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168202    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168207    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168211    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168216    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168221    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168225    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168230    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168234    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168238    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.168243    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169406    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:48:19.170706 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169441    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169449    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169454    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169458    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169464    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169469    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169473    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169478    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169482    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169487    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169491    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169495    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169500    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169504    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169508    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169512    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169517    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169521    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169525    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169529    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:48:19.171154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169533    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169538    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169542    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169546    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169550    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169554    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169558    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169562    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169566    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169571    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169576    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169580    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169587    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169592    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169597    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169602    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169607    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169612    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169616    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169620    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:48:19.171715 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169625    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169629    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169633    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169637    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169642    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169647    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169651    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169656    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169661    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169666    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169670    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169674    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169678    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169682    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169686    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169690    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169695    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169699    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169703    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169707    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:48:19.172199 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169711    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169715    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169750    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169765    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169769    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169773    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169776    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169780    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169784    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169788    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169791    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169794    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169797    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169800    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169802    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169805    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169808    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169811    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169813    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169816    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:48:19.172707 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169818    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169821    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169823    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169826    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169828    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169830    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169833    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169836    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169838    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169841    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169843    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169845    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169848    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169850    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169853    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169863    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169867    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.169869    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170264    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:48:19.173191 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170269    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170272    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170274    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170277    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170279    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170282    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170284    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170287    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170290    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170292    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170295    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170297    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170300    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170302    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170305    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170307    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170310    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170313    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170315    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:48:19.173677 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170318    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170320    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170322    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170325    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170327    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170330    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170332    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170335    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170337    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170341    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170344    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170348    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170351    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170354    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170357    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170360    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170362    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170366    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170370    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:48:19.174145 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170373    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170376    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170378    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170381    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170384    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170386    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170389    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170392    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170395    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170397    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170400    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170402    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170404    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170407    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170409    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170412    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170414    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170416    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170435    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170438    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:48:19.174633 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170441    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170443    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170446    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170449    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170452    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170455    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170457    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170460    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170462    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170465    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170467    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170470    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170472    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170475    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170477    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170480    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170482    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170485    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170487    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170491    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:48:19.175107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170494    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170497    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170500    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170503    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170505    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170507    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170510    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170512    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170515    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170517    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170520    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170522    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170524    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170527    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170529    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170532    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170535    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170538    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170540    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.170543    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170613    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 23:48:19.175606 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170621    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170627    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170632    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170636    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170639    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170644    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170648    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170651    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170654    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170657    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170661    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170664    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170667    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170670    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170672    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170675    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170678    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170682    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170684    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170687    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170690    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170693    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170698    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 23:48:19.176117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170700    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170704    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170707    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170711    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170713    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170717    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170720    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170723    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170727    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170730    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170732    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170735    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170738    2564 flags.go:64] FLAG: --enable-server="true"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170740    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170745    2564 flags.go:64] FLAG: --event-burst="100"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170748    2564 flags.go:64] FLAG: --event-qps="50"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170751    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170753    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170756    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170759    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170762    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170765    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170768    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170771    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170774    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 23:48:19.176726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170776    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170779    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170782    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170784    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170787    2564 flags.go:64] FLAG: --feature-gates=""
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170790    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170793    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170796    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170801    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170805    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170808    2564 flags.go:64] FLAG: --help="false"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170811    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170814    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170817    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170820    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170823    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170826    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170829    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170831    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170834    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170837    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170840    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170843    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170846    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 06 23:48:19.177351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170848    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170851    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170854    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170856    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170859    2564 flags.go:64] FLAG: --lock-file=""
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170862    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170864    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170867    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170872    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170875    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170878    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170881    2564 flags.go:64] FLAG: --logging-format="text"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170883    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170887    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170889    2564 flags.go:64] FLAG: --manifest-url=""
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170892    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170898    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170902    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170906    2564 flags.go:64] FLAG: --max-pods="110"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170909    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170912    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170914    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170917    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170920    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170923    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 23:48:19.177971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170926    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170932    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170935    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170938    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170941    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170944    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170949    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170952    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170955    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170958    2564 flags.go:64] FLAG: --port="10250"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170960    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170963    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c8889beb8cbebe3a"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170966    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170969    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170972    2564 flags.go:64] FLAG: --register-node="true"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170974    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170978    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170980    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170983    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170986    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170990    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170992    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170995    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.170998    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171003    2564 flags.go:64] FLAG: --runonce="false"
Jul 06 23:48:19.178644 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171006    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171009    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171012    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171014    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171017    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171023    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171026    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171029    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171032    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171035    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171038    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171040    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171043    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171046    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171048    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171054    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171056    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171059    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171063    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171066    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171068    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171071    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171074    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171077    2564 flags.go:64] FLAG: --v="2"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171080    2564 flags.go:64] FLAG: --version="false"
Jul 06 23:48:19.179236 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171084    2564 flags.go:64] FLAG: --vmodule=""
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171087    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.171090    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171200    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171203    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171206    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171212    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171215    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171217    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171220    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171223    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171225    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171228    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171231    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171234    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171236    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171239    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171241    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171244    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171246    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:48:19.180020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171248    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171251    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171253    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171256    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171258    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171261    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171263    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171266    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171268    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171270    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171273    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171275    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171278    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171280    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171282    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171285    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171287    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171289    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171295    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171298    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:48:19.180526 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171300    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171303    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171307    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171310    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171313    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171316    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171319    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171321    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171324    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171326    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171328    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171331    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171333    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171336    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171338    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171340    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171343    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171345    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171347    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:48:19.181020 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171351    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171354    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171357    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171360    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171363    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171366    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171368    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171371    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171373    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171376    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171379    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171387    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171391    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171394    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171397    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171400    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171402    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171405    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171408    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:48:19.181522 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171411    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171414    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171416    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171433    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171437    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171439    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171442    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171444    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171447    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171450    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171452    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171455    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171457    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171460    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171462    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171464    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171467    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171469    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171471    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171474    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:48:19.182065 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171476    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171479    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171481    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.171484    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.172025    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.178724    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.178741    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.179415    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.179445    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179505    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179511    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179514    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179517    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179521    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179524    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:48:19.182781 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179527    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179530    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179533    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179535    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179538    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179540    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179543    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179547    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179550    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179553    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179556    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179560    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179564    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179568    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179571    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179573    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179576    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179578    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179581    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:48:19.183154 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179584    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179586    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179589    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179592    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179595    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179598    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179600    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179603    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179606    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179609    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179612    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179614    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179617    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179620    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179622    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179625    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179627    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179630    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179632    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179634    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:48:19.183651 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179637    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179639    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179642    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179652    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179655    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179658    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179660    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179663    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179665    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179668    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179670    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179672    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179675    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179678    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179680    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179683    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179686    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179689    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179692    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179694    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:48:19.184142 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179697    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179700    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179702    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179705    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179707    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179711    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179714    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179718    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179722    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179724    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179727    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179730    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179732    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179735    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179737    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179740    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179742    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179745    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179747    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:48:19.184648 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179750    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179752    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179755    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179757    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179759    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179762    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179765    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179767    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179770    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179773    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179784    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179786    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179789    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179792    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179794    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:48:19.185107 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.179800    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179923    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179928    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179931    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179935    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179938    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179941    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179944    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179947    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179949    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179952    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179954    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179957    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179960    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179963    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179967    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179970    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179974    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179976    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179979    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:48:19.185494 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179982    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179985    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179987    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179990    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179993    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179995    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.179998    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180001    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180004    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180006    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180009    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180011    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180014    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180016    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180018    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180021    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180023    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180026    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180029    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:48:19.185971 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180031    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180034    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180037    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180040    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180042    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180045    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180047    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180050    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180052    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180054    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180057    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180059    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180062    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180064    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180066    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180069    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180071    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180074    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180076    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180078    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:48:19.186503 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180081    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180084    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180087    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180089    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180091    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180094    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180096    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180099    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180101    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180103    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180106    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180109    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180111    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180113    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180116    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180119    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180121    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180125    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180128    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:48:19.186976 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180131    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180133    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180136    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180139    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180141    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180143    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180146    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180148    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180150    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180153    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180156    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180158    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180160    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180163    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180166    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180168    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180171    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180173    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180175    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180178    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:48:19.187442 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180180    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:19.180182    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.180187    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.180301    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.183805    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.185028    2564 server.go:1019] "Starting client certificate rotation"
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.185130    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.185170    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:48:19.187940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.187378    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:48:19.190121 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.190100    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:48:19.199931 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.199913    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 06 23:48:19.201900 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.201886    2564 log.go:25] "Validated CRI v1 image API"
Jul 06 23:48:19.202764 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.202748    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 23:48:19.204933 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.204915    2564 fs.go:135] Filesystem UUIDs: map[58a0c177-654e-4648-b7d7-e9ab86a7a994:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fc982a8c-6cec-42c8-a986-64b99bd1d849:/dev/nvme0n1p3]
Jul 06 23:48:19.204994 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.204933    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 23:48:19.210654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.210542    2564 manager.go:216] Machine: {Timestamp:2026-07-06 23:48:19.209410957 +0000 UTC m=+0.274742924 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095411 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cfb7293448d363510ed6f98b8fccf SystemUUID:ec2cfb72-9344-8d36-3510-ed6f98b8fccf BootID:69ba39c9-351f-4c56-ad62-f3cc2a6415fc Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b8:51:7c:a4:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b8:51:7c:a4:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:b5:ef:79:25:e5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 23:48:19.210654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.210648    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 23:48:19.210751 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.210727    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 23:48:19.213539 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.213509    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 23:48:19.213674 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.213543    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-92.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 23:48:19.214225 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.214215    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 23:48:19.214265 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.214228    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 23:48:19.214265 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.214235    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:48:19.214376 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.214364    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:48:19.214878 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.214865    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:48:19.216109 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.216098    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 23:48:19.218827 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.218815    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:48:19.218933 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.218924    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 23:48:19.218962 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.218953    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 23:48:19.218989 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.218962    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 23:48:19.218989 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.218980    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 23:48:19.219050 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.218990    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 06 23:48:19.219050 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.218999    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 23:48:19.219356 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.219344    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:48:19.219408 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.219363    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:48:19.222944 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.222312    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 23:48:19.224641 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.224624    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 23:48:19.224734 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.224663    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 23:48:19.225675 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225662    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225682    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225692    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225700    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225712    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225720    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225731    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225739    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 23:48:19.225749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225749    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 23:48:19.226001 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225757    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 23:48:19.226001 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225770    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 23:48:19.226001 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.225779    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 23:48:19.226414 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.226403    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 23:48:19.226486 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.226416    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 23:48:19.229685 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.229670    2564 server.go:1268] "Started kubelet"
Jul 06 23:48:19.229796 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.229766    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 23:48:19.229880 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.229802    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 23:48:19.229989 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.229904    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 23:48:19.230657 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.230626    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 23:48:19.230749 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.230671    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 23:48:19.230749 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.230691    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-92.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 23:48:19.230792 ip-10-0-130-92 systemd[1]: Started Kubernetes Kubelet.
Jul 06 23:48:19.231192 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.231072    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 23:48:19.232200 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.232187    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 23:48:19.234866 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.234835    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 23:48:19.234938 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.234162    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-92.ec2.internal.18bfd834e983f300  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-92.ec2.internal,UID:ip-10-0-130-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-92.ec2.internal,},FirstTimestamp:2026-07-06 23:48:19.22964352 +0000 UTC m=+0.294975489,LastTimestamp:2026-07-06 23:48:19.22964352 +0000 UTC m=+0.294975489,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-92.ec2.internal,}"
Jul 06 23:48:19.236972 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.236956    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 23:48:19.237051 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.236973    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 23:48:19.237097 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.237054    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.237671 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.237644    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 23:48:19.238777 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.238759    2564 factory.go:55] Registering systemd factory
Jul 06 23:48:19.238867 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.238800    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 06 23:48:19.239252 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239236    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4drd8"
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239449    2564 factory.go:153] Registering CRI-O factory
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239460    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239478    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239465    2564 factory.go:223] Registration of the crio container factory successfully
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239591    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239617    2564 factory.go:103] Registering Raw factory
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239632    2564 manager.go:1196] Started watching for new ooms in manager
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239692    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.239705    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 23:48:19.240134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.240125    2564 manager.go:349] Starting recovery of all containers
Jul 06 23:48:19.241539 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.241388    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 23:48:19.241706 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.241687    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 23:48:19.247365 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.247330    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4drd8"
Jul 06 23:48:19.254083 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.254066    2564 manager.go:354] Recovery completed
Jul 06 23:48:19.257926 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.257913    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:48:19.259883 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.259869    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:48:19.259947 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.259897    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:48:19.259947 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.259907    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:48:19.260434 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.260408    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 23:48:19.260483 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.260455    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 23:48:19.260483 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.260465    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:48:19.260483 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.260472    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:48:19.261104 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.261088    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:48:19.261153 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.261117    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:48:19.261153 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.261127    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:48:19.261658 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.261647    2564 policy_none.go:49] "None policy: Start"
Jul 06 23:48:19.261692 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.261662    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 23:48:19.261692 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.261672    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 23:48:19.262105 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.262044    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-92.ec2.internal.18bfd834eb51626a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-92.ec2.internal,UID:ip-10-0-130-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-92.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-92.ec2.internal,},FirstTimestamp:2026-07-06 23:48:19.259884138 +0000 UTC m=+0.325216107,LastTimestamp:2026-07-06 23:48:19.259884138 +0000 UTC m=+0.325216107,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-92.ec2.internal,}"
Jul 06 23:48:19.262612 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.262601    2564 policy_none.go:47] "Start"
Jul 06 23:48:19.294995 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.294979    2564 manager.go:337] "Starting Device Plugin manager"
Jul 06 23:48:19.295071 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.295010    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 23:48:19.295071 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.295022    2564 server.go:87] "Starting device plugin registration server"
Jul 06 23:48:19.295276 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.295262    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 23:48:19.295342 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.295276    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 23:48:19.295390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.295366    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 23:48:19.295498 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.295480    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 23:48:19.295498 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.295496    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 23:48:19.295978 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.295958    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 23:48:19.296048 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.296015    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.383314 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.383267    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 23:48:19.384635 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.384615    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 23:48:19.384719 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.384648    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 23:48:19.384719 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.384673    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 23:48:19.384719 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.384705    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 23:48:19.387113 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.387090    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:48:19.395769 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.395755    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:48:19.396668 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.396653    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:48:19.396758 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.396683    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:48:19.396758 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.396694    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:48:19.396758 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.396720    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.405936 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.405922    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.405982 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.405943    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-92.ec2.internal\": node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.422665 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.422610    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.485470 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.485441    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal"]
Jul 06 23:48:19.485551 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.485529    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:48:19.486404 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.486386    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:48:19.486513 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.486418    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:48:19.486513 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.486449    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:48:19.487766 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.487753    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:48:19.487911 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.487898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.487963 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.487925    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:48:19.488452 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.488436    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:48:19.488502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.488465    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:48:19.488502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.488474    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:48:19.488571 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.488436    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:48:19.488571 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.488545    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:48:19.488571 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.488557    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:48:19.489805 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.489790    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.489847 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.489819    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:48:19.490480 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.490467    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:48:19.490546 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.490491    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:48:19.490546 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.490503    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:48:19.516110 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.516090    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-92.ec2.internal\" not found" node="ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.520770 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.520755    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-92.ec2.internal\" not found" node="ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.522782 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.522768    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.540994 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.540975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b574a658b6026d17b7636034f150900-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal\" (UID: \"2b574a658b6026d17b7636034f150900\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.541062 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.541000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b574a658b6026d17b7636034f150900-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal\" (UID: \"2b574a658b6026d17b7636034f150900\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.541062 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.541015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f4db311b901a006fd4141a884f57806-config\") pod \"kube-apiserver-proxy-ip-10-0-130-92.ec2.internal\" (UID: \"9f4db311b901a006fd4141a884f57806\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.623310 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.623281    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.641924 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.641900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f4db311b901a006fd4141a884f57806-config\") pod \"kube-apiserver-proxy-ip-10-0-130-92.ec2.internal\" (UID: \"9f4db311b901a006fd4141a884f57806\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.642007 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.641931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b574a658b6026d17b7636034f150900-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal\" (UID: \"2b574a658b6026d17b7636034f150900\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.642007 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.641951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b574a658b6026d17b7636034f150900-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal\" (UID: \"2b574a658b6026d17b7636034f150900\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.642007 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.641996    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f4db311b901a006fd4141a884f57806-config\") pod \"kube-apiserver-proxy-ip-10-0-130-92.ec2.internal\" (UID: \"9f4db311b901a006fd4141a884f57806\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.642097 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.642008    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b574a658b6026d17b7636034f150900-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal\" (UID: \"2b574a658b6026d17b7636034f150900\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.642129 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.642086    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b574a658b6026d17b7636034f150900-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal\" (UID: \"2b574a658b6026d17b7636034f150900\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.724340 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.724267    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.818939 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.818903    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.823668 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:19.823648    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:19.824731 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.824710    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:19.925304 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:19.925265    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:20.025978 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:20.025903    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:20.126499 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:20.126469    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:20.141725 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.141700    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:48:20.184511 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.184480    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 23:48:20.185085 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.184595    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 23:48:20.185085 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.184631    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 23:48:20.226970 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:20.226942    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:20.252440 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.252390    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:43:19 +0000 UTC" deadline="2028-03-09 09:05:32.800330304 +0000 UTC"
Jul 06 23:48:20.252440 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.252435    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14673h17m12.547915048s"
Jul 06 23:48:20.299229 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:20.298939    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4db311b901a006fd4141a884f57806.slice/crio-ea90d9646bc638f4c8acfdc072ceb145ccdf6b30640a0bca5ddb0c3815ab60e7 WatchSource:0}: Error finding container ea90d9646bc638f4c8acfdc072ceb145ccdf6b30640a0bca5ddb0c3815ab60e7: Status 404 returned error can't find the container with id ea90d9646bc638f4c8acfdc072ceb145ccdf6b30640a0bca5ddb0c3815ab60e7
Jul 06 23:48:20.299229 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:20.299172    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b574a658b6026d17b7636034f150900.slice/crio-8253193b5e92230238c1738773f37f6ec470b1ba79991ff57651a9ff84446dcd WatchSource:0}: Error finding container 8253193b5e92230238c1738773f37f6ec470b1ba79991ff57651a9ff84446dcd: Status 404 returned error can't find the container with id 8253193b5e92230238c1738773f37f6ec470b1ba79991ff57651a9ff84446dcd
Jul 06 23:48:20.303484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.303470    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:48:20.327842 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:20.327811    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-92.ec2.internal\" not found"
Jul 06 23:48:20.340400 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.340383    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:48:20.388797 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.388750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal" event={"ID":"9f4db311b901a006fd4141a884f57806","Type":"ContainerStarted","Data":"ea90d9646bc638f4c8acfdc072ceb145ccdf6b30640a0bca5ddb0c3815ab60e7"}
Jul 06 23:48:20.389590 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.389575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal" event={"ID":"2b574a658b6026d17b7636034f150900","Type":"ContainerStarted","Data":"8253193b5e92230238c1738773f37f6ec470b1ba79991ff57651a9ff84446dcd"}
Jul 06 23:48:20.437516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.437498    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:20.448508 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.448488    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:48:20.449900 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.449886    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal"
Jul 06 23:48:20.459550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.459535    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:48:20.571652 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:20.571579    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:48:21.171355 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.171326    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:48:21.219881 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.219840    2564 apiserver.go:52] "Watching apiserver"
Jul 06 23:48:21.227537 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.227515    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 23:48:21.229159 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.229138    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-gkjd4","openshift-cluster-node-tuning-operator/tuned-jhqg9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal","openshift-multus/network-metrics-daemon-56p5q","openshift-network-diagnostics/network-check-target-gl4f5","openshift-ovn-kubernetes/ovnkube-node-s9mv7","kube-system/global-pull-secret-syncer-s4bnc","kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh","openshift-dns/node-resolver-sgk8d","openshift-image-registry/node-ca-cqmp5","openshift-multus/multus-8nck4","openshift-multus/multus-additional-cni-plugins-49rpw","openshift-network-operator/iptables-alerter-vdspm"]
Jul 06 23:48:21.231685 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.231652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.231787 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.231763    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:21.233526 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.233512    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.235982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.235953    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.236479 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.236457    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 23:48:21.236570 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.236460    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 23:48:21.236666 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.236649    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 23:48:21.236876 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.236859    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 23:48:21.236946 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.236908    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 23:48:21.237179 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.237023    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8nlrp\""
Jul 06 23:48:21.237985 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.237968    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:21.238071 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.237968    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 23:48:21.238182 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.238035    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:21.238942 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.238897    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:48:21.239152 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.239134    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6hpdl\""
Jul 06 23:48:21.239246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.239149    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 23:48:21.240029 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.240010    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:21.240126 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.240070    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:21.242217 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.242200    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 23:48:21.243027 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.242401    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.244392 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.244377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.245023 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.244976    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 23:48:21.245126 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.245090    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 23:48:21.245126 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.245096    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 23:48:21.245473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.245301    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 23:48:21.245473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.245349    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hqs45\""
Jul 06 23:48:21.246154 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.246135    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 23:48:21.246377 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.246239    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 23:48:21.246741 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.246725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.248059 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.247617    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 23:48:21.248059 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.247701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 23:48:21.248059 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.247917    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 23:48:21.248569 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.248552    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bnj6b\""
Jul 06 23:48:21.249108 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.248981    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.249395    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qcmn4\""
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.249541    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.249775    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.249956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-os-release\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.249984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250013    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88ba113d-1b1a-4d9e-b803-d3e783643d1f-host-slash\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250035    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9c4c5715-f342-45cd-a938-c93cf23382b5-kubelet-config\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250054    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-system-cni-dir\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-cnibin\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250093    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-cni-binary-copy\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250114    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86vnf\" (UniqueName: \"kubernetes.io/projected/0fd3b1e7-7b3d-436b-b950-23cda301c067-kube-api-access-86vnf\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2h77\" (UniqueName: \"kubernetes.io/projected/94f2f896-2166-460e-9cec-8cb2e977fa92-kube-api-access-f2h77\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9c4c5715-f342-45cd-a938-c93cf23382b5-dbus\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.251390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57brw\" (UniqueName: \"kubernetes.io/projected/88ba113d-1b1a-4d9e-b803-d3e783643d1f-kube-api-access-57brw\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.252245 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.252245 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88ba113d-1b1a-4d9e-b803-d3e783643d1f-iptables-alerter-script\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.252245 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250331    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:21.252245 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.250955    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.260065 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.259531    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:48:21.260065 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.259569    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 23:48:21.260065 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.259757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.260278 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.260134    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 23:48:21.260278 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.260237    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wrd4j\""
Jul 06 23:48:21.260364 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.260191    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 23:48:21.260446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.260360    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p2wq7\""
Jul 06 23:48:21.261909 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.261889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.263545 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.263522    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:48:21.263545 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.263543    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 23:48:21.263760 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.263742    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 23:48:21.263843 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.263744    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 23:48:21.264166 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.264141    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z4bjs\""
Jul 06 23:48:21.264742 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.264723    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 23:48:21.266753 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.266734    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5lzfl\""
Jul 06 23:48:21.283308 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.283287    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tql7p"
Jul 06 23:48:21.294541 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.294522    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tql7p"
Jul 06 23:48:21.339910 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.339886    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 23:48:21.350976 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.350949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-sys\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.351117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.350980    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-env-overrides\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxmmx\" (UniqueName: \"kubernetes.io/projected/70c9a419-4f23-46fa-be19-bc75d9d97011-kube-api-access-xxmmx\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.351117 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-log-socket\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-device-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.351272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.351272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351195    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8fc4d22f-e22b-4597-b7ff-572393b0154c-cni-binary-copy\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.351272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-netns\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.351272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-ovn\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysconfig\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38d42341-576a-44ab-9d70-95558dae56cb-etc-tuned\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351323    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-cnibin\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-cni-binary-copy\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351376    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-cnibin\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-etc-kubernetes\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-systemd-units\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-etc-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70c9a419-4f23-46fa-be19-bc75d9d97011-ovn-node-metrics-cert\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-ovnkube-script-lib\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351542    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2h77\" (UniqueName: \"kubernetes.io/projected/94f2f896-2166-460e-9cec-8cb2e977fa92-kube-api-access-f2h77\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9c4c5715-f342-45cd-a938-c93cf23382b5-dbus\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5bc2\" (UniqueName: \"kubernetes.io/projected/8fc4d22f-e22b-4597-b7ff-572393b0154c-kube-api-access-f5bc2\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.351595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-cni-netd\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-socket-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351628    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysctl-conf\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351644    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-run\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351660    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-host\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57brw\" (UniqueName: \"kubernetes.io/projected/88ba113d-1b1a-4d9e-b803-d3e783643d1f-kube-api-access-57brw\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-system-cni-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351702    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2r76\" (UniqueName: \"kubernetes.io/projected/38d42341-576a-44ab-9d70-95558dae56cb-kube-api-access-j2r76\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1048959-28a5-4e60-922b-4c6af41cafc7-tmp-dir\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351735    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e740413-5fb8-490c-8254-dfc1d73a84ac-serviceca\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-cni-multus\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-kubelet\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351804    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-ovnkube-config\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351836    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-cni-binary-copy\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351846    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-sys-fs\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351910    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fd3b1e7-7b3d-436b-b950-23cda301c067-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.352191 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.351940    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9c4c5715-f342-45cd-a938-c93cf23382b5-dbus\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-var-lib-kubelet\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-k8s-cni-cncf-io\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352062    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-node-log\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-registration-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-socket-dir-parent\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-kubernetes\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352170    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352220    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-cnibin\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-hostroot\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-slash\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.352271    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352285    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-systemd\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90ba0ca7-eb97-4dbd-acff-67537d724463-agent-certs\") pod \"konnectivity-agent-gkjd4\" (UID: \"90ba0ca7-eb97-4dbd-acff-67537d724463\") " pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.352333    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret podName:9c4c5715-f342-45cd-a938-c93cf23382b5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:21.852313086 +0000 UTC m=+2.917645039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret") pod "global-pull-secret-syncer-s4bnc" (UID: "9c4c5715-f342-45cd-a938-c93cf23382b5") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:21.353005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-systemd\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-lib-modules\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88ba113d-1b1a-4d9e-b803-d3e783643d1f-iptables-alerter-script\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352567    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-os-release\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352744    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-os-release\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352910    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d9mz\" (UniqueName: \"kubernetes.io/projected/7e740413-5fb8-490c-8254-dfc1d73a84ac-kube-api-access-7d9mz\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352936    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-cni-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-os-release\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352976    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-run-netns\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352991    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88ba113d-1b1a-4d9e-b803-d3e783643d1f-iptables-alerter-script\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.352994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1048959-28a5-4e60-922b-4c6af41cafc7-hosts-file\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88ba113d-1b1a-4d9e-b803-d3e783643d1f-host-slash\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-system-cni-dir\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86vnf\" (UniqueName: \"kubernetes.io/projected/0fd3b1e7-7b3d-436b-b950-23cda301c067-kube-api-access-86vnf\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353123    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-system-cni-dir\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.353709 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-conf-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353155    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88ba113d-1b1a-4d9e-b803-d3e783643d1f-host-slash\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-multus-certs\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353207    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-var-lib-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353272    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-cni-bin\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-cni-bin\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpkwd\" (UniqueName: \"kubernetes.io/projected/d1048959-28a5-4e60-922b-4c6af41cafc7-kube-api-access-lpkwd\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e740413-5fb8-490c-8254-dfc1d73a84ac-host\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353358    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-kubelet\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.353379    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353399    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fd3b1e7-7b3d-436b-b950-23cda301c067-tuning-conf-dir\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353379    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.353451    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs podName:94f2f896-2166-460e-9cec-8cb2e977fa92 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:21.853433989 +0000 UTC m=+2.918765956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs") pod "network-metrics-daemon-56p5q" (UID: "94f2f896-2166-460e-9cec-8cb2e977fa92") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysctl-d\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353513    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnb42\" (UniqueName: \"kubernetes.io/projected/46397cec-c19b-4548-abe4-c218b60d334a-kube-api-access-tnb42\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.354221 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353536    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90ba0ca7-eb97-4dbd-acff-67537d724463-konnectivity-ca\") pod \"konnectivity-agent-gkjd4\" (UID: \"90ba0ca7-eb97-4dbd-acff-67537d724463\") " pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.354705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353560    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-modprobe-d\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.354705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353579    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38d42341-576a-44ab-9d70-95558dae56cb-tmp\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.354705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9c4c5715-f342-45cd-a938-c93cf23382b5-kubelet-config\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.354705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-daemon-config\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.354705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.353699    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9c4c5715-f342-45cd-a938-c93cf23382b5-kubelet-config\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.365556 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.365533    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:48:21.365672 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.365570    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:48:21.365672 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.365580    2564 projected.go:196] Error preparing data for projected volume kube-api-access-bdv8g for pod openshift-network-diagnostics/network-check-target-gl4f5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:21.365672 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.365629    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g podName:0b21f861-6631-4376-8b6e-1980034ea780 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:21.865615267 +0000 UTC m=+2.930947221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bdv8g" (UniqueName: "kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g") pod "network-check-target-gl4f5" (UID: "0b21f861-6631-4376-8b6e-1980034ea780") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:21.365898 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.365872    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 23:48:21.368997 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.368966    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86vnf\" (UniqueName: \"kubernetes.io/projected/0fd3b1e7-7b3d-436b-b950-23cda301c067-kube-api-access-86vnf\") pod \"multus-additional-cni-plugins-49rpw\" (UID: \"0fd3b1e7-7b3d-436b-b950-23cda301c067\") " pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.368997 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.368993    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57brw\" (UniqueName: \"kubernetes.io/projected/88ba113d-1b1a-4d9e-b803-d3e783643d1f-kube-api-access-57brw\") pod \"iptables-alerter-vdspm\" (UID: \"88ba113d-1b1a-4d9e-b803-d3e783643d1f\") " pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.371286 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.371263    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2h77\" (UniqueName: \"kubernetes.io/projected/94f2f896-2166-460e-9cec-8cb2e977fa92-kube-api-access-f2h77\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:21.454726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-system-cni-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.454726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2r76\" (UniqueName: \"kubernetes.io/projected/38d42341-576a-44ab-9d70-95558dae56cb-kube-api-access-j2r76\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.454726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1048959-28a5-4e60-922b-4c6af41cafc7-tmp-dir\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.454726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454727    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e740413-5fb8-490c-8254-dfc1d73a84ac-serviceca\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454748    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-cni-multus\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-kubelet\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-ovnkube-config\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-sys-fs\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-var-lib-kubelet\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454835    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-k8s-cni-cncf-io\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-node-log\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-registration-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-socket-dir-parent\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-kubernetes\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-cnibin\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.454950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-hostroot\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454968    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-slash\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-systemd\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.454999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90ba0ca7-eb97-4dbd-acff-67537d724463-agent-certs\") pod \"konnectivity-agent-gkjd4\" (UID: \"90ba0ca7-eb97-4dbd-acff-67537d724463\") " pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-systemd\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-lib-modules\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d9mz\" (UniqueName: \"kubernetes.io/projected/7e740413-5fb8-490c-8254-dfc1d73a84ac-kube-api-access-7d9mz\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-cni-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-os-release\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-run-netns\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1048959-28a5-4e60-922b-4c6af41cafc7-hosts-file\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-conf-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-multus-certs\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455170    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-var-lib-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455193    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-cni-bin\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455235    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-cni-bin\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpkwd\" (UniqueName: \"kubernetes.io/projected/d1048959-28a5-4e60-922b-4c6af41cafc7-kube-api-access-lpkwd\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e740413-5fb8-490c-8254-dfc1d73a84ac-host\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.455374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-kubelet\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysctl-d\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnb42\" (UniqueName: \"kubernetes.io/projected/46397cec-c19b-4548-abe4-c218b60d334a-kube-api-access-tnb42\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90ba0ca7-eb97-4dbd-acff-67537d724463-konnectivity-ca\") pod \"konnectivity-agent-gkjd4\" (UID: \"90ba0ca7-eb97-4dbd-acff-67537d724463\") " pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-modprobe-d\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455406    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38d42341-576a-44ab-9d70-95558dae56cb-tmp\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455440    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-daemon-config\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-sys\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-env-overrides\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxmmx\" (UniqueName: \"kubernetes.io/projected/70c9a419-4f23-46fa-be19-bc75d9d97011-kube-api-access-xxmmx\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-log-socket\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455547    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-device-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455576    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8fc4d22f-e22b-4597-b7ff-572393b0154c-cni-binary-copy\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-netns\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456053 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455635    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-ovn\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455690    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysconfig\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38d42341-576a-44ab-9d70-95558dae56cb-etc-tuned\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455720    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-run-netns\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-etc-kubernetes\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455777    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-etc-kubernetes\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455786    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-system-cni-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455825    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1048959-28a5-4e60-922b-4c6af41cafc7-hosts-file\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455846    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-multus-certs\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455856    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-conf-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455888    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-var-lib-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455926    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-cni-bin\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.455960    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-cni-bin\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456132    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1048959-28a5-4e60-922b-4c6af41cafc7-tmp-dir\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456213    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e740413-5fb8-490c-8254-dfc1d73a84ac-host\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456253    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-kubelet\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456291    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.456713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456362    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-cni-multus\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456397    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-var-lib-kubelet\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456452    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysctl-d\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456847    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e740413-5fb8-490c-8254-dfc1d73a84ac-serviceca\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456869    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-ovnkube-config\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456926    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-var-lib-kubelet\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.456965    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-k8s-cni-cncf-io\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457003    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-node-log\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457044    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457088    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-registration-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457131    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-socket-dir-parent\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457176    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-kubernetes\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457220    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-cnibin\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457256    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-hostroot\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457733 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457293    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-slash\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.457733 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457328    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-systemd\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.457733 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457546    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-systemd\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.457733 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457610    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-run-ovn-kubernetes\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.457733 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457707    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-modprobe-d\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.457897 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457740    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-lib-modules\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.457897 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457813    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-os-release\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457897 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457864    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-cni-dir\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.457897 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-systemd-units\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458015 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.457935    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-systemd-units\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458156 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458134    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.458234 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458206    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-env-overrides\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458270 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-etc-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458305 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458274    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70c9a419-4f23-46fa-be19-bc75d9d97011-ovn-node-metrics-cert\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458305 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458298    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-ovnkube-script-lib\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458389 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5bc2\" (UniqueName: \"kubernetes.io/projected/8fc4d22f-e22b-4597-b7ff-572393b0154c-kube-api-access-f5bc2\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.458389 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-cni-netd\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458389 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458372    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-sys\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.458389 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458380    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-socket-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.458548 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysctl-conf\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.458548 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-run\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.458548 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-host\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.458548 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458541    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-host\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.458712 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458577    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-etc-openvswitch\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458874 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458850    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8fc4d22f-e22b-4597-b7ff-572393b0154c-multus-daemon-config\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.458941 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458906    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-cni-netd\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.458941 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.458920    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-log-socket\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.459055 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459034    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysctl-conf\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.459137 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459117    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-socket-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.459385 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459334    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-run-ovn\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.459385 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459362    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70c9a419-4f23-46fa-be19-bc75d9d97011-ovnkube-script-lib\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.459385 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459398    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-device-dir\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.459621 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459454    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8fc4d22f-e22b-4597-b7ff-572393b0154c-host-run-netns\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.459621 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459452    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70c9a419-4f23-46fa-be19-bc75d9d97011-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.459621 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459496    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/46397cec-c19b-4548-abe4-c218b60d334a-sys-fs\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.459621 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459516    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-etc-sysconfig\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.459621 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459538    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38d42341-576a-44ab-9d70-95558dae56cb-run\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.459855 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459796    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8fc4d22f-e22b-4597-b7ff-572393b0154c-cni-binary-copy\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.459961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.459942    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90ba0ca7-eb97-4dbd-acff-67537d724463-konnectivity-ca\") pod \"konnectivity-agent-gkjd4\" (UID: \"90ba0ca7-eb97-4dbd-acff-67537d724463\") " pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.460085 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.460056    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38d42341-576a-44ab-9d70-95558dae56cb-tmp\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.460260 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.460238    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90ba0ca7-eb97-4dbd-acff-67537d724463-agent-certs\") pod \"konnectivity-agent-gkjd4\" (UID: \"90ba0ca7-eb97-4dbd-acff-67537d724463\") " pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.460958 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.460937    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70c9a419-4f23-46fa-be19-bc75d9d97011-ovn-node-metrics-cert\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.461472 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.461452    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38d42341-576a-44ab-9d70-95558dae56cb-etc-tuned\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.464168 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.464151    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpkwd\" (UniqueName: \"kubernetes.io/projected/d1048959-28a5-4e60-922b-4c6af41cafc7-kube-api-access-lpkwd\") pod \"node-resolver-sgk8d\" (UID: \"d1048959-28a5-4e60-922b-4c6af41cafc7\") " pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.466350 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.466323    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2r76\" (UniqueName: \"kubernetes.io/projected/38d42341-576a-44ab-9d70-95558dae56cb-kube-api-access-j2r76\") pod \"tuned-jhqg9\" (UID: \"38d42341-576a-44ab-9d70-95558dae56cb\") " pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.467523 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.467500    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d9mz\" (UniqueName: \"kubernetes.io/projected/7e740413-5fb8-490c-8254-dfc1d73a84ac-kube-api-access-7d9mz\") pod \"node-ca-cqmp5\" (UID: \"7e740413-5fb8-490c-8254-dfc1d73a84ac\") " pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.467620 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.467520    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxmmx\" (UniqueName: \"kubernetes.io/projected/70c9a419-4f23-46fa-be19-bc75d9d97011-kube-api-access-xxmmx\") pod \"ovnkube-node-s9mv7\" (UID: \"70c9a419-4f23-46fa-be19-bc75d9d97011\") " pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.468033 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.468014    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnb42\" (UniqueName: \"kubernetes.io/projected/46397cec-c19b-4548-abe4-c218b60d334a-kube-api-access-tnb42\") pod \"aws-ebs-csi-driver-node-6mfbh\" (UID: \"46397cec-c19b-4548-abe4-c218b60d334a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.468432 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.468398    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5bc2\" (UniqueName: \"kubernetes.io/projected/8fc4d22f-e22b-4597-b7ff-572393b0154c-kube-api-access-f5bc2\") pod \"multus-8nck4\" (UID: \"8fc4d22f-e22b-4597-b7ff-572393b0154c\") " pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.546249 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.546219    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-49rpw"
Jul 06 23:48:21.563275 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.563251    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdspm"
Jul 06 23:48:21.571013 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.570992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:21.576702 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.576682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh"
Jul 06 23:48:21.582378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.582359    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:21.588772 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.588746    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jhqg9"
Jul 06 23:48:21.595457 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.595440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sgk8d"
Jul 06 23:48:21.600784 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.600763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cqmp5"
Jul 06 23:48:21.606250 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.606233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8nck4"
Jul 06 23:48:21.861840 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.861760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:21.861840 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.861811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:21.862023 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.861901    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:21.862023 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.861932    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:21.862023 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.861966    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret podName:9c4c5715-f342-45cd-a938-c93cf23382b5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:22.861948069 +0000 UTC m=+3.927280025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret") pod "global-pull-secret-syncer-s4bnc" (UID: "9c4c5715-f342-45cd-a938-c93cf23382b5") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:21.862023 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.861986    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs podName:94f2f896-2166-460e-9cec-8cb2e977fa92 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:22.861976644 +0000 UTC m=+3.927308598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs") pod "network-metrics-daemon-56p5q" (UID: "94f2f896-2166-460e-9cec-8cb2e977fa92") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:21.962448 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:21.962400    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:21.962583 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.962529    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:48:21.962583 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.962548    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:48:21.962583 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.962557    2564 projected.go:196] Error preparing data for projected volume kube-api-access-bdv8g for pod openshift-network-diagnostics/network-check-target-gl4f5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:21.962710 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:21.962605    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g podName:0b21f861-6631-4376-8b6e-1980034ea780 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:22.962591394 +0000 UTC m=+4.027923380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdv8g" (UniqueName: "kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g") pod "network-check-target-gl4f5" (UID: "0b21f861-6631-4376-8b6e-1980034ea780") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:21.998217 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:21.997999    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c9a419_4f23_46fa_be19_bc75d9d97011.slice/crio-8b9b31d6a3dd0e6b3a91f8d0c0c596f117aa7045d0d01af7e4e7c7f1fb14d0ef WatchSource:0}: Error finding container 8b9b31d6a3dd0e6b3a91f8d0c0c596f117aa7045d0d01af7e4e7c7f1fb14d0ef: Status 404 returned error can't find the container with id 8b9b31d6a3dd0e6b3a91f8d0c0c596f117aa7045d0d01af7e4e7c7f1fb14d0ef
Jul 06 23:48:21.998688 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:21.998661    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc4d22f_e22b_4597_b7ff_572393b0154c.slice/crio-f6e28e12fe99adadd59be9e70dbbfc52896c9e7c70112c7126cf89e10fd7f2f5 WatchSource:0}: Error finding container f6e28e12fe99adadd59be9e70dbbfc52896c9e7c70112c7126cf89e10fd7f2f5: Status 404 returned error can't find the container with id f6e28e12fe99adadd59be9e70dbbfc52896c9e7c70112c7126cf89e10fd7f2f5
Jul 06 23:48:21.999788 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:21.999770    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ba113d_1b1a_4d9e_b803_d3e783643d1f.slice/crio-afb8111b892ae1177124bd94fac84b71de37cb41427bcb8be2862bff97fff090 WatchSource:0}: Error finding container afb8111b892ae1177124bd94fac84b71de37cb41427bcb8be2862bff97fff090: Status 404 returned error can't find the container with id afb8111b892ae1177124bd94fac84b71de37cb41427bcb8be2862bff97fff090
Jul 06 23:48:22.001576 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:22.001188    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d42341_576a_44ab_9d70_95558dae56cb.slice/crio-98f07477816e8fa6b526841af82018a9e65373ef8e4f05989e657acfa8a0fc45 WatchSource:0}: Error finding container 98f07477816e8fa6b526841af82018a9e65373ef8e4f05989e657acfa8a0fc45: Status 404 returned error can't find the container with id 98f07477816e8fa6b526841af82018a9e65373ef8e4f05989e657acfa8a0fc45
Jul 06 23:48:22.004115 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:22.004096    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e740413_5fb8_490c_8254_dfc1d73a84ac.slice/crio-5d34fcab2b04a54a2f1e03a4a1822d56dd8427cc0b785368f8ad9517a13f73c0 WatchSource:0}: Error finding container 5d34fcab2b04a54a2f1e03a4a1822d56dd8427cc0b785368f8ad9517a13f73c0: Status 404 returned error can't find the container with id 5d34fcab2b04a54a2f1e03a4a1822d56dd8427cc0b785368f8ad9517a13f73c0
Jul 06 23:48:22.005200 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:22.005182    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46397cec_c19b_4548_abe4_c218b60d334a.slice/crio-951e99829ed08488168795ea01ab0b750e42b99f2c58a42e7a19b23e875bdd33 WatchSource:0}: Error finding container 951e99829ed08488168795ea01ab0b750e42b99f2c58a42e7a19b23e875bdd33: Status 404 returned error can't find the container with id 951e99829ed08488168795ea01ab0b750e42b99f2c58a42e7a19b23e875bdd33
Jul 06 23:48:22.006344 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:22.006322    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1048959_28a5_4e60_922b_4c6af41cafc7.slice/crio-4191f82432f550889ebe12491b0a4f8274be8f42dc8352a1055a8e9373873f59 WatchSource:0}: Error finding container 4191f82432f550889ebe12491b0a4f8274be8f42dc8352a1055a8e9373873f59: Status 404 returned error can't find the container with id 4191f82432f550889ebe12491b0a4f8274be8f42dc8352a1055a8e9373873f59
Jul 06 23:48:22.009705 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:22.008235    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ba0ca7_eb97_4dbd_acff_67537d724463.slice/crio-1f9c4d3cb48e09b684ddcda0a0306fcef7de19acb0610e56af7b90b3d8e56a19 WatchSource:0}: Error finding container 1f9c4d3cb48e09b684ddcda0a0306fcef7de19acb0610e56af7b90b3d8e56a19: Status 404 returned error can't find the container with id 1f9c4d3cb48e09b684ddcda0a0306fcef7de19acb0610e56af7b90b3d8e56a19
Jul 06 23:48:22.009705 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:48:22.009466    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fd3b1e7_7b3d_436b_b950_23cda301c067.slice/crio-ba1d9726cb6485c1e924f7dc28c732de51f99a6acbf51eefd431365e25ee0e2b WatchSource:0}: Error finding container ba1d9726cb6485c1e924f7dc28c732de51f99a6acbf51eefd431365e25ee0e2b: Status 404 returned error can't find the container with id ba1d9726cb6485c1e924f7dc28c732de51f99a6acbf51eefd431365e25ee0e2b
Jul 06 23:48:22.297476 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.297396    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:21 +0000 UTC" deadline="2028-03-02 09:19:15.724366843 +0000 UTC"
Jul 06 23:48:22.297476 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.297468    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14505h30m53.426900934s"
Jul 06 23:48:22.398600 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.398554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cqmp5" event={"ID":"7e740413-5fb8-490c-8254-dfc1d73a84ac","Type":"ContainerStarted","Data":"5d34fcab2b04a54a2f1e03a4a1822d56dd8427cc0b785368f8ad9517a13f73c0"}
Jul 06 23:48:22.399803 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.399777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jhqg9" event={"ID":"38d42341-576a-44ab-9d70-95558dae56cb","Type":"ContainerStarted","Data":"98f07477816e8fa6b526841af82018a9e65373ef8e4f05989e657acfa8a0fc45"}
Jul 06 23:48:22.401367 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.401317    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdspm" event={"ID":"88ba113d-1b1a-4d9e-b803-d3e783643d1f","Type":"ContainerStarted","Data":"afb8111b892ae1177124bd94fac84b71de37cb41427bcb8be2862bff97fff090"}
Jul 06 23:48:22.404527 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.404498    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"8b9b31d6a3dd0e6b3a91f8d0c0c596f117aa7045d0d01af7e4e7c7f1fb14d0ef"}
Jul 06 23:48:22.406724 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.406696    2564 generic.go:356] "Generic (PLEG): container finished" podID="2b574a658b6026d17b7636034f150900" containerID="9058daef4e3107d6e5b90fb66ad78ddd1c4557265a033257d6d407c6e97a37dd" exitCode=0
Jul 06 23:48:22.406819 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.406779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal" event={"ID":"2b574a658b6026d17b7636034f150900","Type":"ContainerDied","Data":"9058daef4e3107d6e5b90fb66ad78ddd1c4557265a033257d6d407c6e97a37dd"}
Jul 06 23:48:22.408734 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.408679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerStarted","Data":"ba1d9726cb6485c1e924f7dc28c732de51f99a6acbf51eefd431365e25ee0e2b"}
Jul 06 23:48:22.411005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.410975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gkjd4" event={"ID":"90ba0ca7-eb97-4dbd-acff-67537d724463","Type":"ContainerStarted","Data":"1f9c4d3cb48e09b684ddcda0a0306fcef7de19acb0610e56af7b90b3d8e56a19"}
Jul 06 23:48:22.413551 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.413504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sgk8d" event={"ID":"d1048959-28a5-4e60-922b-4c6af41cafc7","Type":"ContainerStarted","Data":"4191f82432f550889ebe12491b0a4f8274be8f42dc8352a1055a8e9373873f59"}
Jul 06 23:48:22.415465 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.415437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8nck4" event={"ID":"8fc4d22f-e22b-4597-b7ff-572393b0154c","Type":"ContainerStarted","Data":"f6e28e12fe99adadd59be9e70dbbfc52896c9e7c70112c7126cf89e10fd7f2f5"}
Jul 06 23:48:22.417444 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.417315    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal" event={"ID":"9f4db311b901a006fd4141a884f57806","Type":"ContainerStarted","Data":"ac6d78b2a3aa20732e87ccb2535afb2479ddecea6d79f144d4c7706a95629f52"}
Jul 06 23:48:22.420048 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.420029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh" event={"ID":"46397cec-c19b-4548-abe4-c218b60d334a","Type":"ContainerStarted","Data":"951e99829ed08488168795ea01ab0b750e42b99f2c58a42e7a19b23e875bdd33"}
Jul 06 23:48:22.444857 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.444800    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-92.ec2.internal" podStartSLOduration=2.444781078 podStartE2EDuration="2.444781078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:22.444641794 +0000 UTC m=+3.509973771" watchObservedRunningTime="2026-07-06 23:48:22.444781078 +0000 UTC m=+3.510113055"
Jul 06 23:48:22.870149 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.869349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:22.870149 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.869438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:22.870149 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.869584    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:22.870149 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.869646    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs podName:94f2f896-2166-460e-9cec-8cb2e977fa92 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:24.869626129 +0000 UTC m=+5.934958103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs") pod "network-metrics-daemon-56p5q" (UID: "94f2f896-2166-460e-9cec-8cb2e977fa92") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:22.870149 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.870018    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:22.870149 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.870063    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret podName:9c4c5715-f342-45cd-a938-c93cf23382b5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:24.870048848 +0000 UTC m=+5.935380817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret") pod "global-pull-secret-syncer-s4bnc" (UID: "9c4c5715-f342-45cd-a938-c93cf23382b5") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:22.925012 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.924818    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:48:22.970762 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:22.970724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:22.970939 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.970911    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:48:22.970939 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.970933    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:48:22.971050 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.970946    2564 projected.go:196] Error preparing data for projected volume kube-api-access-bdv8g for pod openshift-network-diagnostics/network-check-target-gl4f5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:22.971050 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:22.971005    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g podName:0b21f861-6631-4376-8b6e-1980034ea780 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:24.970987669 +0000 UTC m=+6.036319639 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdv8g" (UniqueName: "kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g") pod "network-check-target-gl4f5" (UID: "0b21f861-6631-4376-8b6e-1980034ea780") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:23.300621 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:23.300528    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:21 +0000 UTC" deadline="2028-03-18 21:58:58.989659634 +0000 UTC"
Jul 06 23:48:23.300621 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:23.300572    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14902h10m35.689092306s"
Jul 06 23:48:23.386622 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:23.386585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:23.386786 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:23.386714    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:23.387145 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:23.387128    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:23.387285 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:23.387236    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:23.388943 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:23.388917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:23.389052 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:23.389013    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:23.443332 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:23.443269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal" event={"ID":"2b574a658b6026d17b7636034f150900","Type":"ContainerStarted","Data":"dea6c485e302f0895cd7417999e027be82a981c153869256f9bcc13e211e15ac"}
Jul 06 23:48:23.461146 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:23.460572    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-92.ec2.internal" podStartSLOduration=3.460556246 podStartE2EDuration="3.460556246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:23.460487819 +0000 UTC m=+4.525819797" watchObservedRunningTime="2026-07-06 23:48:23.460556246 +0000 UTC m=+4.525888224"
Jul 06 23:48:24.889108 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:24.889073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:24.889664 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:24.889143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:24.889664 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.889284    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:24.889664 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.889344    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs podName:94f2f896-2166-460e-9cec-8cb2e977fa92 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:28.889325918 +0000 UTC m=+9.954657877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs") pod "network-metrics-daemon-56p5q" (UID: "94f2f896-2166-460e-9cec-8cb2e977fa92") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:24.889999 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.889963    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:24.890075 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.890017    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret podName:9c4c5715-f342-45cd-a938-c93cf23382b5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:28.890001416 +0000 UTC m=+9.955333377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret") pod "global-pull-secret-syncer-s4bnc" (UID: "9c4c5715-f342-45cd-a938-c93cf23382b5") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:24.990076 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:24.990027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:24.990256 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.990238    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:48:24.990328 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.990264    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:48:24.990328 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.990277    2564 projected.go:196] Error preparing data for projected volume kube-api-access-bdv8g for pod openshift-network-diagnostics/network-check-target-gl4f5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:24.990476 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:24.990349    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g podName:0b21f861-6631-4376-8b6e-1980034ea780 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:28.990317126 +0000 UTC m=+10.055649100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdv8g" (UniqueName: "kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g") pod "network-check-target-gl4f5" (UID: "0b21f861-6631-4376-8b6e-1980034ea780") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:25.385521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:25.385349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:25.385521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:25.385396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:25.385521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:25.385498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:25.385737 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:25.385522    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:25.385737 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:25.385607    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:25.385737 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:25.385717    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:27.387493 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:27.387457    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:27.388022 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:27.387598    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:27.388022 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:27.387977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:27.388121 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:27.388088    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:27.388199 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:27.388158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:27.388247 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:27.388229    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:28.922947 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:28.922903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:28.923446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:28.922991    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:28.923446 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:28.923150    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:28.923446 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:28.923209    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs podName:94f2f896-2166-460e-9cec-8cb2e977fa92 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:36.923194191 +0000 UTC m=+17.988526146 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs") pod "network-metrics-daemon-56p5q" (UID: "94f2f896-2166-460e-9cec-8cb2e977fa92") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:28.923763 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:28.923745    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:28.923832 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:28.923795    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret podName:9c4c5715-f342-45cd-a938-c93cf23382b5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:36.923780591 +0000 UTC m=+17.989112546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret") pod "global-pull-secret-syncer-s4bnc" (UID: "9c4c5715-f342-45cd-a938-c93cf23382b5") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:29.023897 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:29.023860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:29.024111 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:29.024033    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:48:29.024111 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:29.024056    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:48:29.024111 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:29.024066    2564 projected.go:196] Error preparing data for projected volume kube-api-access-bdv8g for pod openshift-network-diagnostics/network-check-target-gl4f5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:29.024275 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:29.024119    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g podName:0b21f861-6631-4376-8b6e-1980034ea780 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:37.024106344 +0000 UTC m=+18.089438298 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdv8g" (UniqueName: "kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g") pod "network-check-target-gl4f5" (UID: "0b21f861-6631-4376-8b6e-1980034ea780") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:29.385541 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:29.385511    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:29.385738 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:29.385639    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:29.387026 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:29.386813    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:29.387026 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:29.386922    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:29.387026 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:29.386961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:29.387026 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:29.387001    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:31.385985 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:31.385946    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:31.386440 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:31.386079    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:31.386541 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:31.386516    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:31.386662 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:31.386623    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:31.386734 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:31.386672    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:31.386790 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:31.386736    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:33.384955 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:33.384922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:33.385453 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:33.384922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:33.385453 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:33.385057    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:33.385453 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:33.385078    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:33.385453 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:33.385192    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:33.385453 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:33.385291    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:35.386003 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:35.385970    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:35.386486 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:35.385966    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:35.386486 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:35.386115    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:35.386486 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:35.386134    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:35.386486 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:35.386238    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:35.386486 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:35.386308    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:36.981492 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:36.981441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:36.981492 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:36.981499    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:36.982057 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:36.981617    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:36.982057 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:36.981618    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:36.982057 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:36.981684    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs podName:94f2f896-2166-460e-9cec-8cb2e977fa92 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:52.981663254 +0000 UTC m=+34.046995226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs") pod "network-metrics-daemon-56p5q" (UID: "94f2f896-2166-460e-9cec-8cb2e977fa92") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:36.982057 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:36.981702    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret podName:9c4c5715-f342-45cd-a938-c93cf23382b5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:52.981692822 +0000 UTC m=+34.047024777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret") pod "global-pull-secret-syncer-s4bnc" (UID: "9c4c5715-f342-45cd-a938-c93cf23382b5") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:37.082816 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:37.082782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:37.083004 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:37.082970    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:48:37.083004 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:37.082988    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:48:37.083004 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:37.082997    2564 projected.go:196] Error preparing data for projected volume kube-api-access-bdv8g for pod openshift-network-diagnostics/network-check-target-gl4f5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:37.083117 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:37.083054    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g podName:0b21f861-6631-4376-8b6e-1980034ea780 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:53.08303801 +0000 UTC m=+34.148369987 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdv8g" (UniqueName: "kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g") pod "network-check-target-gl4f5" (UID: "0b21f861-6631-4376-8b6e-1980034ea780") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:37.385623 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:37.384876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:37.385623 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:37.384912    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:37.385623 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:37.385027    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:37.385623 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:37.385441    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:37.385623 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:37.385493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:37.385623 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:37.385550    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:39.386516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:39.386482    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:39.386995 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:39.386594    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:39.386995 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:39.386613    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:39.386995 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:39.386794    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:39.386995 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:39.386821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:39.386995 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:39.386933    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:41.385493 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.385185    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:41.385493 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:41.385463    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:41.386506 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.385509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:41.386506 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:41.385619    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:41.386506 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.385665    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:41.386506 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:41.385724    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:41.488542 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.488513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh" event={"ID":"46397cec-c19b-4548-abe4-c218b60d334a","Type":"ContainerStarted","Data":"bdf7e683797139e43175639d64e8ad86ae2bec4ccbc65477da589c7e38e7c13d"}
Jul 06 23:48:41.490156 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.490132    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cqmp5" event={"ID":"7e740413-5fb8-490c-8254-dfc1d73a84ac","Type":"ContainerStarted","Data":"d9c57533b7bae8cf79ae34a299901c1c2636b4b9f5e1ae7feed58ba27539c092"}
Jul 06 23:48:41.496148 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.496099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jhqg9" event={"ID":"38d42341-576a-44ab-9d70-95558dae56cb","Type":"ContainerStarted","Data":"5f330012f97d83f825a9aefbde9338eeeb65ea420467c4732b6bd8d1bfd0bf12"}
Jul 06 23:48:41.498000 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.497974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"7731aaca8e39080024908336819f5f1947f5db75900f0b260427ef4e452f5a4d"}
Jul 06 23:48:41.499552 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.499527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerStarted","Data":"7fb6cd0b88633699ef6cfc6840aa4b7ad11eed0eb53ae96786b7137d2a8ba11b"}
Jul 06 23:48:41.501204 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.501172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gkjd4" event={"ID":"90ba0ca7-eb97-4dbd-acff-67537d724463","Type":"ContainerStarted","Data":"caf497152657351a8d8970071c4f543c1be17222f2a24ca92f6066fc7c1e9c99"}
Jul 06 23:48:41.502441 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.502405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sgk8d" event={"ID":"d1048959-28a5-4e60-922b-4c6af41cafc7","Type":"ContainerStarted","Data":"0c3f0ee5d37db3746fabd0b00ecc889a49d78d0697f2e48fa431656f2aa0b9d5"}
Jul 06 23:48:41.503825 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.503805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8nck4" event={"ID":"8fc4d22f-e22b-4597-b7ff-572393b0154c","Type":"ContainerStarted","Data":"192d1a1a8c315330df56d861fd02e9e867e898286f92f195495f3ae9b8ecfb89"}
Jul 06 23:48:41.542209 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.542157    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cqmp5" podStartSLOduration=3.4646664 podStartE2EDuration="22.54214098s" totalImagesPullingTime="19.07747458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:41.514343374 +0000 UTC m=+22.579675352" watchObservedRunningTime="2026-07-06 23:48:41.54214098 +0000 UTC m=+22.607472957"
Jul 06 23:48:41.571235 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.571188    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gkjd4" podStartSLOduration=3.497808101 podStartE2EDuration="22.571151377s" totalImagesPullingTime="19.073343276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:41.543925522 +0000 UTC m=+22.609257498" watchObservedRunningTime="2026-07-06 23:48:41.571151377 +0000 UTC m=+22.636483353"
Jul 06 23:48:41.611218 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.611035    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sgk8d" podStartSLOduration=3.537000896 podStartE2EDuration="22.611019291s" totalImagesPullingTime="19.074018395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:41.571255088 +0000 UTC m=+22.636587068" watchObservedRunningTime="2026-07-06 23:48:41.611019291 +0000 UTC m=+22.676351267"
Jul 06 23:48:41.642275 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:41.642217    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jhqg9" podStartSLOduration=3.550760545 podStartE2EDuration="22.64219791s" totalImagesPullingTime="19.091437365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:41.642108139 +0000 UTC m=+22.707440114" watchObservedRunningTime="2026-07-06 23:48:41.64219791 +0000 UTC m=+22.707529888"
Jul 06 23:48:42.106766 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.106674    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cqmp5_7e740413-5fb8-490c-8254-dfc1d73a84ac/node-ca/0.log"
Jul 06 23:48:42.508488 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.508461    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 06 23:48:42.508914 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.508766    2564 generic.go:356] "Generic (PLEG): container finished" podID="70c9a419-4f23-46fa-be19-bc75d9d97011" containerID="d9955d2f2a56c9867387bdd0d92b09aa8809a199407db6112c45f5a9bb9ddf6b" exitCode=1
Jul 06 23:48:42.508914 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.508829    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"1872fb01e759f0dfa7fdcbdc6bb12b8d5d924801bc1a51d5ac25a6dcf35cd859"}
Jul 06 23:48:42.511557 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.509242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"a13434d765f942350daec384297dc74a260e64974c6f542d985bd1b2f3805afe"}
Jul 06 23:48:42.511557 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.509474    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"6e1d162bbc4ba823805b1956c1c4091bb080101866861c0002cd06c3fcc5f973"}
Jul 06 23:48:42.511557 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.509552    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"1a5ac6a31d43f4b7de3aeebf3d8da3c62ac42b39b1d8ec3a6a01c63d1282d82c"}
Jul 06 23:48:42.511557 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.509621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerDied","Data":"d9955d2f2a56c9867387bdd0d92b09aa8809a199407db6112c45f5a9bb9ddf6b"}
Jul 06 23:48:42.512849 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.512825    2564 generic.go:356] "Generic (PLEG): container finished" podID="0fd3b1e7-7b3d-436b-b950-23cda301c067" containerID="7fb6cd0b88633699ef6cfc6840aa4b7ad11eed0eb53ae96786b7137d2a8ba11b" exitCode=0
Jul 06 23:48:42.512959 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.512924    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerDied","Data":"7fb6cd0b88633699ef6cfc6840aa4b7ad11eed0eb53ae96786b7137d2a8ba11b"}
Jul 06 23:48:42.550500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.550458    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8nck4" podStartSLOduration=4.447366774 podStartE2EDuration="23.550442716s" totalImagesPullingTime="19.103075942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:41.673386374 +0000 UTC m=+22.738718350" watchObservedRunningTime="2026-07-06 23:48:42.550442716 +0000 UTC m=+23.615774683"
Jul 06 23:48:42.856021 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.855997    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 23:48:42.985345 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.985142    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:42.985671 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:42.985656    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:43.314828 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.314738    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:48:42.856014984Z","UUID":"6f472dfe-82c4-42ac-b2e5-3799c61ffc0f","Handler":null,"Name":"","Endpoint":""}
Jul 06 23:48:43.316169 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.316151    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 23:48:43.316307 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.316190    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 23:48:43.385410 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.385376    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:43.385410 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.385400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:43.385705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.385529    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:43.385705 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:43.385603    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:43.385705 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:43.385676    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:43.385705 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:43.385542    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:43.517680 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.517597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh" event={"ID":"46397cec-c19b-4548-abe4-c218b60d334a","Type":"ContainerStarted","Data":"1b95992426e5841186d754f25e132b63e1eb6fe3bcd4e9bef77352b704033b52"}
Jul 06 23:48:43.519337 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.519165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdspm" event={"ID":"88ba113d-1b1a-4d9e-b803-d3e783643d1f","Type":"ContainerStarted","Data":"edf3ea8d0506e5e9fe77ad51d69baf67889be0e108f6c4d7168b3bac67ca6f37"}
Jul 06 23:48:43.519337 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.519217    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:43.520357 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.520281    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gkjd4"
Jul 06 23:48:43.534599 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:43.534362    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vdspm" podStartSLOduration=5.436931384 podStartE2EDuration="24.534341448s" totalImagesPullingTime="19.097410064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:43.533467764 +0000 UTC m=+24.598799739" watchObservedRunningTime="2026-07-06 23:48:43.534341448 +0000 UTC m=+24.599673426"
Jul 06 23:48:44.524092 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:44.524047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh" event={"ID":"46397cec-c19b-4548-abe4-c218b60d334a","Type":"ContainerStarted","Data":"d3eb02e575ab236610c9bfc6dc726674243e893b450b3d5e33a74fd587af1441"}
Jul 06 23:48:44.527141 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:44.527122    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 06 23:48:44.527447 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:44.527412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"110c56d3a919fb92507da1ddd515c22b96c32615852fdbc478480876f6c2063a"}
Jul 06 23:48:44.543624 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:44.543578    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mfbh" podStartSLOduration=3.785991327 podStartE2EDuration="25.543565201s" totalImagesPullingTime="21.757573874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:44.54208982 +0000 UTC m=+25.607421796" watchObservedRunningTime="2026-07-06 23:48:44.543565201 +0000 UTC m=+25.608897224"
Jul 06 23:48:45.385872 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:45.385836    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:45.386161 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:45.385971    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:45.386400 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:45.386375    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:45.386549 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:45.386499    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:45.386549 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:45.386527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:45.386662 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:45.386583    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:47.385373 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:47.385343    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:47.386112 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:47.385352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:47.386112 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:47.385493    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:47.386112 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:47.385352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:47.386112 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:47.385551    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:47.386112 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:47.385591    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:48.539583 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.539386    2564 generic.go:356] "Generic (PLEG): container finished" podID="0fd3b1e7-7b3d-436b-b950-23cda301c067" containerID="a10e389139cd91553395d3c2f88c7835d31cbfd5336458a797843c3854c7f903" exitCode=0
Jul 06 23:48:48.540057 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.539469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerDied","Data":"a10e389139cd91553395d3c2f88c7835d31cbfd5336458a797843c3854c7f903"}
Jul 06 23:48:48.543065 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.543048    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 06 23:48:48.543397 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.543378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"8fc406ede4cf107d408eb7280a10c140c763ad618061a75d5791e1e17db210fe"}
Jul 06 23:48:48.543693 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.543670    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:48.543786 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.543747    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:48.543838 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.543808    2564 scope.go:117] "RemoveContainer" containerID="d9955d2f2a56c9867387bdd0d92b09aa8809a199407db6112c45f5a9bb9ddf6b"
Jul 06 23:48:48.560393 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:48.560372    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:49.386371 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.386321    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:49.386371 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.386354    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:49.386694 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.386476    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:49.386694 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:49.386506    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:49.386694 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:49.386552    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:49.386694 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:49.386615    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:49.549298 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.549269    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 06 23:48:49.549702 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.549603    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" event={"ID":"70c9a419-4f23-46fa-be19-bc75d9d97011","Type":"ContainerStarted","Data":"10190d9871def22db2c5d71f3e51f0e165dac20000f82448e3e3576ac0510c27"}
Jul 06 23:48:49.549777 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.549764    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:49.566020 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.565856    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:48:49.581220 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:49.581167    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7" podStartSLOduration=11.271534697 podStartE2EDuration="30.581153131s" totalImagesPullingTime="19.309618434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:49.580750807 +0000 UTC m=+30.646082784" watchObservedRunningTime="2026-07-06 23:48:49.581153131 +0000 UTC m=+30.646485153"
Jul 06 23:48:50.553871 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:50.553839    2564 generic.go:356] "Generic (PLEG): container finished" podID="0fd3b1e7-7b3d-436b-b950-23cda301c067" containerID="64ee34da32a0b9adbc4656bb1b5fda92524fba0ea73f79332d03ad0e07a441ba" exitCode=0
Jul 06 23:48:50.554252 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:50.553923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerDied","Data":"64ee34da32a0b9adbc4656bb1b5fda92524fba0ea73f79332d03ad0e07a441ba"}
Jul 06 23:48:51.384984 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:51.384944    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:51.385179 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:51.384960    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:51.385179 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:51.385076    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:51.385179 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:51.385094    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:51.385294 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:51.385209    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:51.385328 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:51.385293    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:52.560144 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:52.560110    2564 generic.go:356] "Generic (PLEG): container finished" podID="0fd3b1e7-7b3d-436b-b950-23cda301c067" containerID="34c808e8c39789ee13c49995cec7c39c2dcea0e8151010a9db22ebb806d3b5b5" exitCode=0
Jul 06 23:48:52.560524 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:52.560156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerDied","Data":"34c808e8c39789ee13c49995cec7c39c2dcea0e8151010a9db22ebb806d3b5b5"}
Jul 06 23:48:53.010050 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:53.010010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:53.010222 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:53.010064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:53.010222 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.010159    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:53.010222 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.010161    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:53.010222 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.010209    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs podName:94f2f896-2166-460e-9cec-8cb2e977fa92 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:25.01019644 +0000 UTC m=+66.075528394 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs") pod "network-metrics-daemon-56p5q" (UID: "94f2f896-2166-460e-9cec-8cb2e977fa92") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:48:53.010222 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.010222    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret podName:9c4c5715-f342-45cd-a938-c93cf23382b5 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:25.010216178 +0000 UTC m=+66.075548133 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret") pod "global-pull-secret-syncer-s4bnc" (UID: "9c4c5715-f342-45cd-a938-c93cf23382b5") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:48:53.111072 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:53.111038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:53.111245 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.111169    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:48:53.111245 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.111192    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:48:53.111245 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.111205    2564 projected.go:196] Error preparing data for projected volume kube-api-access-bdv8g for pod openshift-network-diagnostics/network-check-target-gl4f5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:53.111405 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.111258    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g podName:0b21f861-6631-4376-8b6e-1980034ea780 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:25.111245046 +0000 UTC m=+66.176577001 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bdv8g" (UniqueName: "kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g") pod "network-check-target-gl4f5" (UID: "0b21f861-6631-4376-8b6e-1980034ea780") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:48:53.385546 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:53.385379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:53.385546 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:53.385406    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:53.385546 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:53.385452    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:53.385793 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.385560    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:53.385793 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.385606    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:53.385793 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:53.385641    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:55.385967 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:55.385929    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:55.386490 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:55.385929    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:55.386490 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:55.386061    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:55.386490 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:55.386056    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:55.386490 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:55.386156    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:55.386490 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:55.386246    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:57.385159 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:57.385115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:57.385159 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:57.385152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:57.385763 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:57.385125    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:57.385763 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:57.385218    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:57.385763 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:57.385351    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:57.385763 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:57.385511    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:57.575776 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:57.575742    2564 generic.go:356] "Generic (PLEG): container finished" podID="0fd3b1e7-7b3d-436b-b950-23cda301c067" containerID="78b10c1133dd5c7d06a818ddc12ae3e2357a5424add232d1db122ad924cd2d61" exitCode=0
Jul 06 23:48:57.575926 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:57.575797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerDied","Data":"78b10c1133dd5c7d06a818ddc12ae3e2357a5424add232d1db122ad924cd2d61"}
Jul 06 23:48:58.581092 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:58.581060    2564 generic.go:356] "Generic (PLEG): container finished" podID="0fd3b1e7-7b3d-436b-b950-23cda301c067" containerID="b0c6845075284050d41df17a07e0346469c7e9bcb0447b7310969fdca71eaf84" exitCode=0
Jul 06 23:48:58.581585 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:58.581117    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerDied","Data":"b0c6845075284050d41df17a07e0346469c7e9bcb0447b7310969fdca71eaf84"}
Jul 06 23:48:59.386410 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:59.386377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:48:59.386598 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:59.386483    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:48:59.386640 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:59.386593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:48:59.386738 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:59.386716    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:48:59.386773 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:59.386754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:48:59.386843 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:48:59.386826    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:48:59.586506 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:59.586476    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-49rpw" event={"ID":"0fd3b1e7-7b3d-436b-b950-23cda301c067","Type":"ContainerStarted","Data":"6af95ed3b4250ba833832920c17df4879b9886a76ab217f5fb636a6869725c6a"}
Jul 06 23:48:59.610492 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:48:59.610442    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-49rpw" podStartSLOduration=9.49812175 podStartE2EDuration="40.610408522s" totalImagesPullingTime="31.112286772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:59.609945725 +0000 UTC m=+40.675277712" watchObservedRunningTime="2026-07-06 23:48:59.610408522 +0000 UTC m=+40.675740498"
Jul 06 23:49:01.385389 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:01.385345    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:01.385819 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:01.385365    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:01.385819 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:01.385473    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:49:01.385819 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:01.385558    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:49:01.385819 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:01.385623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:01.385819 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:01.385700    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:49:03.384956 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:03.384917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:03.384956 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:03.384941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:03.385403 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:03.384925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:03.385403 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:03.385040    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:49:03.385403 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:03.385125    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:49:03.385403 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:03.385208    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:49:05.388414 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:05.388381    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:05.388949 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:05.388498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:05.388949 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:05.388514    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:49:05.388949 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:05.388586    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:49:05.388949 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:05.388621    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:05.388949 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:05.388675    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:49:06.224003 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:06.223820    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s4bnc"]
Jul 06 23:49:06.224143 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:06.224132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:06.224246 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:06.224223    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:49:06.224812 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:06.224782    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gl4f5"]
Jul 06 23:49:06.225051 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:06.225034    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:06.225163 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:06.225142    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:49:06.225823 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:06.225756    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-56p5q"]
Jul 06 23:49:06.225900 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:06.225886    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:06.225993 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:06.225975    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:49:07.385878 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:07.385836    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:07.386413 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:07.386000    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:49:08.385215 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:08.385178    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:08.385358 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:08.385198    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:08.385358 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:08.385318    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-56p5q" podUID="94f2f896-2166-460e-9cec-8cb2e977fa92"
Jul 06 23:49:08.385462 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:08.385361    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s4bnc" podUID="9c4c5715-f342-45cd-a938-c93cf23382b5"
Jul 06 23:49:09.388075 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.388046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:09.388482 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:09.388134    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl4f5" podUID="0b21f861-6631-4376-8b6e-1980034ea780"
Jul 06 23:49:09.826478 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.826418    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-92.ec2.internal" event="NodeReady"
Jul 06 23:49:09.826638 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.826604    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 23:49:09.869447 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.869399    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nl57w"]
Jul 06 23:49:09.903690 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.903652    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7g28g"]
Jul 06 23:49:09.904011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.903839    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:09.907008 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.906981    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 23:49:09.907123 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.907043    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5f2wr\""
Jul 06 23:49:09.907123 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.907054    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 23:49:09.916295 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.916278    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nl57w"]
Jul 06 23:49:09.916390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.916309    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7g28g"]
Jul 06 23:49:09.916390 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.916324    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s8jnd"]
Jul 06 23:49:09.916474 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.916394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:09.918916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.918900    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wqknp\""
Jul 06 23:49:09.919019 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.918936    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 23:49:09.919078 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.919022    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 23:49:09.919177 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.919159    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 23:49:09.933027 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.933005    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d25186-1b3b-4fcd-a596-6592eb4507ab-config-volume\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:09.933143 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.933031    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58128f39-84e1-43a5-bcdd-91dbdbb82ee2-cert\") pod \"ingress-canary-7g28g\" (UID: \"58128f39-84e1-43a5-bcdd-91dbdbb82ee2\") " pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:09.933143 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.933051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8d25186-1b3b-4fcd-a596-6592eb4507ab-metrics-tls\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:09.933143 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.933070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvl4j\" (UniqueName: \"kubernetes.io/projected/a8d25186-1b3b-4fcd-a596-6592eb4507ab-kube-api-access-zvl4j\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:09.933346 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.933193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d25186-1b3b-4fcd-a596-6592eb4507ab-tmp-dir\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:09.933346 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.933241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7svc\" (UniqueName: \"kubernetes.io/projected/58128f39-84e1-43a5-bcdd-91dbdbb82ee2-kube-api-access-w7svc\") pod \"ingress-canary-7g28g\" (UID: \"58128f39-84e1-43a5-bcdd-91dbdbb82ee2\") " pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:09.934487 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.934468    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s8jnd"]
Jul 06 23:49:09.934593 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.934578    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:09.937235 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.937217    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gz82q\""
Jul 06 23:49:09.937235 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.937230    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 23:49:09.937376 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.937244    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 23:49:09.937376 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.937279    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 23:49:09.937376 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:09.937230    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 23:49:10.034042 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d25186-1b3b-4fcd-a596-6592eb4507ab-config-volume\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.034042 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58128f39-84e1-43a5-bcdd-91dbdbb82ee2-cert\") pod \"ingress-canary-7g28g\" (UID: \"58128f39-84e1-43a5-bcdd-91dbdbb82ee2\") " pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:10.034246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034064    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kh2c\" (UniqueName: \"kubernetes.io/projected/4d6f7503-aaab-4525-a863-8a3daee84210-kube-api-access-7kh2c\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.034246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8d25186-1b3b-4fcd-a596-6592eb4507ab-metrics-tls\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.034246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvl4j\" (UniqueName: \"kubernetes.io/projected/a8d25186-1b3b-4fcd-a596-6592eb4507ab-kube-api-access-zvl4j\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.034246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6f7503-aaab-4525-a863-8a3daee84210-data-volume\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.034246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d6f7503-aaab-4525-a863-8a3daee84210-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.034246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d25186-1b3b-4fcd-a596-6592eb4507ab-tmp-dir\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.034537 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7svc\" (UniqueName: \"kubernetes.io/projected/58128f39-84e1-43a5-bcdd-91dbdbb82ee2-kube-api-access-w7svc\") pod \"ingress-canary-7g28g\" (UID: \"58128f39-84e1-43a5-bcdd-91dbdbb82ee2\") " pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:10.034537 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d6f7503-aaab-4525-a863-8a3daee84210-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.034537 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d6f7503-aaab-4525-a863-8a3daee84210-crio-socket\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.034681 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034611    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d25186-1b3b-4fcd-a596-6592eb4507ab-tmp-dir\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.034753 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.034735    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d25186-1b3b-4fcd-a596-6592eb4507ab-config-volume\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.038249 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.038232    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8d25186-1b3b-4fcd-a596-6592eb4507ab-metrics-tls\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.038314 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.038268    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58128f39-84e1-43a5-bcdd-91dbdbb82ee2-cert\") pod \"ingress-canary-7g28g\" (UID: \"58128f39-84e1-43a5-bcdd-91dbdbb82ee2\") " pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:10.042118 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.042094    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvl4j\" (UniqueName: \"kubernetes.io/projected/a8d25186-1b3b-4fcd-a596-6592eb4507ab-kube-api-access-zvl4j\") pod \"dns-default-nl57w\" (UID: \"a8d25186-1b3b-4fcd-a596-6592eb4507ab\") " pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.042296 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.042271    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7svc\" (UniqueName: \"kubernetes.io/projected/58128f39-84e1-43a5-bcdd-91dbdbb82ee2-kube-api-access-w7svc\") pod \"ingress-canary-7g28g\" (UID: \"58128f39-84e1-43a5-bcdd-91dbdbb82ee2\") " pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:10.135332 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kh2c\" (UniqueName: \"kubernetes.io/projected/4d6f7503-aaab-4525-a863-8a3daee84210-kube-api-access-7kh2c\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.135332 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135330    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6f7503-aaab-4525-a863-8a3daee84210-data-volume\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.135611 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135348    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d6f7503-aaab-4525-a863-8a3daee84210-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.135611 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d6f7503-aaab-4525-a863-8a3daee84210-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.135611 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d6f7503-aaab-4525-a863-8a3daee84210-crio-socket\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.135742 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135616    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d6f7503-aaab-4525-a863-8a3daee84210-crio-socket\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.135779 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135758    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6f7503-aaab-4525-a863-8a3daee84210-data-volume\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.135972 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.135947    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d6f7503-aaab-4525-a863-8a3daee84210-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.137798 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.137772    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d6f7503-aaab-4525-a863-8a3daee84210-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.143368 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.143347    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kh2c\" (UniqueName: \"kubernetes.io/projected/4d6f7503-aaab-4525-a863-8a3daee84210-kube-api-access-7kh2c\") pod \"insights-runtime-extractor-s8jnd\" (UID: \"4d6f7503-aaab-4525-a863-8a3daee84210\") " pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.214256 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.214221    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:10.224485 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.224456    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7g28g"
Jul 06 23:49:10.242962 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.242933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s8jnd"
Jul 06 23:49:10.385438 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.384991    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:10.385438 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.384992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:10.389135 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.389012    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:49:10.389595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.389313    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:49:10.389840 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.389731    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjdfk\""
Jul 06 23:49:10.436194 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.436030    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nl57w"]
Jul 06 23:49:10.440312 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:10.440279    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d25186_1b3b_4fcd_a596_6592eb4507ab.slice/crio-76cc6f4872f3b9eead4b3f2fa5c79b0f6c8d87b1404790df5d110e0e63002a2a WatchSource:0}: Error finding container 76cc6f4872f3b9eead4b3f2fa5c79b0f6c8d87b1404790df5d110e0e63002a2a: Status 404 returned error can't find the container with id 76cc6f4872f3b9eead4b3f2fa5c79b0f6c8d87b1404790df5d110e0e63002a2a
Jul 06 23:49:10.443514 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.443487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s8jnd"]
Jul 06 23:49:10.449794 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.449721    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7g28g"]
Jul 06 23:49:10.452080 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:10.452058    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58128f39_84e1_43a5_bcdd_91dbdbb82ee2.slice/crio-de3e75784f98ae2b0e262ee59a27ea10f8e9fc71973a51251ad33b87b92c90c1 WatchSource:0}: Error finding container de3e75784f98ae2b0e262ee59a27ea10f8e9fc71973a51251ad33b87b92c90c1: Status 404 returned error can't find the container with id de3e75784f98ae2b0e262ee59a27ea10f8e9fc71973a51251ad33b87b92c90c1
Jul 06 23:49:10.611960 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.611854    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7g28g" event={"ID":"58128f39-84e1-43a5-bcdd-91dbdbb82ee2","Type":"ContainerStarted","Data":"de3e75784f98ae2b0e262ee59a27ea10f8e9fc71973a51251ad33b87b92c90c1"}
Jul 06 23:49:10.612931 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.612905    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8jnd" event={"ID":"4d6f7503-aaab-4525-a863-8a3daee84210","Type":"ContainerStarted","Data":"7f94378510d53309244088a8ee38abc75507358fa71703c04a7a569af2e600bb"}
Jul 06 23:49:10.613877 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:10.613859    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nl57w" event={"ID":"a8d25186-1b3b-4fcd-a596-6592eb4507ab","Type":"ContainerStarted","Data":"76cc6f4872f3b9eead4b3f2fa5c79b0f6c8d87b1404790df5d110e0e63002a2a"}
Jul 06 23:49:11.385325 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.385292    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:11.388916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.388204    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m28zf\""
Jul 06 23:49:11.388916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.388267    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:49:11.388916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.388443    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:49:11.590898 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.590868    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6497d74f5d-gn9hd"]
Jul 06 23:49:11.620695 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.620666    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6497d74f5d-gn9hd"]
Jul 06 23:49:11.620917 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.620794    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.625083    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.625331    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.625598    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.625777    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.625978    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.626533    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s7hfr\""
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.626739    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 23:49:11.627139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.626993    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 23:49:11.647725 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.647653    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-oauth-config\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.647725 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.647691    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-config\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.647909 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.647729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-serving-cert\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.647909 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.647777    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-oauth-serving-cert\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.647909 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.647844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fngk4\" (UniqueName: \"kubernetes.io/projected/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-kube-api-access-fngk4\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.647909 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.647901    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-service-ca\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.748619 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.748581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fngk4\" (UniqueName: \"kubernetes.io/projected/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-kube-api-access-fngk4\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.748809 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.748661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-service-ca\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.748809 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.748752    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-oauth-config\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.748809 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.748777    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-config\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.748935 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.748808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-serving-cert\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.748935 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.748830    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-oauth-serving-cert\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.749807 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.749754    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-oauth-serving-cert\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.750239 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.750213    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-config\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.750360 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.750260    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-service-ca\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.754892 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.754815    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-oauth-config\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.756835 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.756811    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-serving-cert\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.758181 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.758140    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fngk4\" (UniqueName: \"kubernetes.io/projected/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-kube-api-access-fngk4\") pod \"console-6497d74f5d-gn9hd\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") " pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:11.937253 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:11.936898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:12.115794 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:12.115759    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6497d74f5d-gn9hd"]
Jul 06 23:49:12.121689 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:12.121658    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52106a2b_62a9_4ada_b374_d0f70bbd7f7d.slice/crio-c779512e2140378e78923cbe583a13921e70a2ca53ebbb024344098ca36412ce WatchSource:0}: Error finding container c779512e2140378e78923cbe583a13921e70a2ca53ebbb024344098ca36412ce: Status 404 returned error can't find the container with id c779512e2140378e78923cbe583a13921e70a2ca53ebbb024344098ca36412ce
Jul 06 23:49:12.621299 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:12.621258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6497d74f5d-gn9hd" event={"ID":"52106a2b-62a9-4ada-b374-d0f70bbd7f7d","Type":"ContainerStarted","Data":"c779512e2140378e78923cbe583a13921e70a2ca53ebbb024344098ca36412ce"}
Jul 06 23:49:14.228389 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.228348    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"]
Jul 06 23:49:14.235140 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.235113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.239519 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.239221    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6ls7b\""
Jul 06 23:49:14.239519 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.239239    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 23:49:14.239519 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.239262    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 23:49:14.239519 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.239309    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 23:49:14.239519 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.239380    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 06 23:49:14.241488 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.241178    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 06 23:49:14.242700 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.242672    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"]
Jul 06 23:49:14.271484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.271443    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48f76fce-bac3-49e8-9079-86d572fe6cd2-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.271672 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.271517    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48f76fce-bac3-49e8-9079-86d572fe6cd2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.271672 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.271564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48f76fce-bac3-49e8-9079-86d572fe6cd2-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.271672 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.271599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vcsn\" (UniqueName: \"kubernetes.io/projected/48f76fce-bac3-49e8-9079-86d572fe6cd2-kube-api-access-6vcsn\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.372727 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.372686    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48f76fce-bac3-49e8-9079-86d572fe6cd2-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.372900 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.372740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vcsn\" (UniqueName: \"kubernetes.io/projected/48f76fce-bac3-49e8-9079-86d572fe6cd2-kube-api-access-6vcsn\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.372900 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.372853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48f76fce-bac3-49e8-9079-86d572fe6cd2-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.373011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.372900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48f76fce-bac3-49e8-9079-86d572fe6cd2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.378438 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.373977    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48f76fce-bac3-49e8-9079-86d572fe6cd2-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.379351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.379313    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48f76fce-bac3-49e8-9079-86d572fe6cd2-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.382022 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.381995    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48f76fce-bac3-49e8-9079-86d572fe6cd2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.383626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.383592    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vcsn\" (UniqueName: \"kubernetes.io/projected/48f76fce-bac3-49e8-9079-86d572fe6cd2-kube-api-access-6vcsn\") pod \"prometheus-operator-567f7b8b7c-m82tt\" (UID: \"48f76fce-bac3-49e8-9079-86d572fe6cd2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:14.548592 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:14.548517    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"
Jul 06 23:49:16.435233 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:16.435200    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt"]
Jul 06 23:49:17.365120 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:17.365090    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f76fce_bac3_49e8_9079_86d572fe6cd2.slice/crio-5be2dcdeebb40096d31facf47715d7ef8db4bfcbfbef2ea44cac440e715d232d WatchSource:0}: Error finding container 5be2dcdeebb40096d31facf47715d7ef8db4bfcbfbef2ea44cac440e715d232d: Status 404 returned error can't find the container with id 5be2dcdeebb40096d31facf47715d7ef8db4bfcbfbef2ea44cac440e715d232d
Jul 06 23:49:17.638219 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.638126    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8jnd" event={"ID":"4d6f7503-aaab-4525-a863-8a3daee84210","Type":"ContainerStarted","Data":"deeee7cc3c23b2110aaabb57f24db7cf63288c44b5522c8ef32ea1d3a70f07c8"}
Jul 06 23:49:17.639324 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.639292    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt" event={"ID":"48f76fce-bac3-49e8-9079-86d572fe6cd2","Type":"ContainerStarted","Data":"5be2dcdeebb40096d31facf47715d7ef8db4bfcbfbef2ea44cac440e715d232d"}
Jul 06 23:49:17.640669 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.640648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7g28g" event={"ID":"58128f39-84e1-43a5-bcdd-91dbdbb82ee2","Type":"ContainerStarted","Data":"894030e47053b5273e6d8e359b66f8b7fb5a7363018e20ee626e423e7471c3bc"}
Jul 06 23:49:17.642293 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.642272    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nl57w" event={"ID":"a8d25186-1b3b-4fcd-a596-6592eb4507ab","Type":"ContainerStarted","Data":"1a5969a921d07e9c7ade76077defd044bb1482ec491789789f10cf1450e1e288"}
Jul 06 23:49:17.642406 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.642310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nl57w" event={"ID":"a8d25186-1b3b-4fcd-a596-6592eb4507ab","Type":"ContainerStarted","Data":"e430483366fcd71aa858f12d4accdc2111f93e2bb620743604321fc8c8700cfa"}
Jul 06 23:49:17.642406 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.642345    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:17.643879 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.643859    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6497d74f5d-gn9hd" event={"ID":"52106a2b-62a9-4ada-b374-d0f70bbd7f7d","Type":"ContainerStarted","Data":"defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736"}
Jul 06 23:49:17.657279 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.657234    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7g28g" podStartSLOduration=2.8063313340000002 podStartE2EDuration="8.657218932s" totalImagesPullingTime="5.850887598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:17.655691416 +0000 UTC m=+58.721023394" watchObservedRunningTime="2026-07-06 23:49:17.657218932 +0000 UTC m=+58.722550903"
Jul 06 23:49:17.672358 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.672308    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nl57w" podStartSLOduration=1.76214479 podStartE2EDuration="8.672294355s" totalImagesPullingTime="6.910149565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:17.672028998 +0000 UTC m=+58.737360974" watchObservedRunningTime="2026-07-06 23:49:17.672294355 +0000 UTC m=+58.737626332"
Jul 06 23:49:17.691222 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:17.691158    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6497d74f5d-gn9hd" podStartSLOduration=1.428965319 podStartE2EDuration="6.691135264s" totalImagesPullingTime="5.262169945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:17.69016143 +0000 UTC m=+58.755493408" watchObservedRunningTime="2026-07-06 23:49:17.691135264 +0000 UTC m=+58.756467242"
Jul 06 23:49:18.651029 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:18.650989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8jnd" event={"ID":"4d6f7503-aaab-4525-a863-8a3daee84210","Type":"ContainerStarted","Data":"9bfd8b1a3551d1eac5f6a555e5b24f680198c3499a703a064210bea7dddfc810"}
Jul 06 23:49:19.655835 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:19.655797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt" event={"ID":"48f76fce-bac3-49e8-9079-86d572fe6cd2","Type":"ContainerStarted","Data":"cba3b4451821312757f961a5016e99d562fa49fd09ec123a090b7cc8b2f88e37"}
Jul 06 23:49:19.656278 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:19.655899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt" event={"ID":"48f76fce-bac3-49e8-9079-86d572fe6cd2","Type":"ContainerStarted","Data":"92dd2fab4f7f88ade9a559ae6b69908bd95a124a9fbe804f9a7618bf2ab199b0"}
Jul 06 23:49:19.673927 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:19.673872    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-m82tt" podStartSLOduration=4.16688742 podStartE2EDuration="5.673852837s" totalImagesPullingTime="1.506965417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:19.673086068 +0000 UTC m=+60.738418045" watchObservedRunningTime="2026-07-06 23:49:19.673852837 +0000 UTC m=+60.739184814"
Jul 06 23:49:21.567382 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.567344    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s9mv7"
Jul 06 23:49:21.582085 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.582058    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"]
Jul 06 23:49:21.583985 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.583957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.586632 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.586612    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5kchp\""
Jul 06 23:49:21.586944 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.586926    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 06 23:49:21.587016 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.586930    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 06 23:49:21.588663 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.588647    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fsmg9"]
Jul 06 23:49:21.590275 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.590257    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 06 23:49:21.590702 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.590685    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.594929 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.594911    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 23:49:21.595017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.594943    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 23:49:21.595306 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.595292    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 23:49:21.595403 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.595379    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q9bcb\""
Jul 06 23:49:21.599058 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.599036    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"]
Jul 06 23:49:21.641041 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qthng\" (UniqueName: \"kubernetes.io/projected/7bfa3930-3794-45ea-94ba-3b019691282f-kube-api-access-qthng\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.641201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-textfile\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-accelerators-collector-config\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.641351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-metrics-client-ca\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwkpj\" (UniqueName: \"kubernetes.io/projected/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-kube-api-access-zwkpj\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-wtmp\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641384    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.641502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7bfa3930-3794-45ea-94ba-3b019691282f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.641502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641482    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-sys\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641496    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-tls\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641643 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-root\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.641696 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bfa3930-3794-45ea-94ba-3b019691282f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.641696 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.641683    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.667169 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.666701    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8jnd" event={"ID":"4d6f7503-aaab-4525-a863-8a3daee84210","Type":"ContainerStarted","Data":"b167f9abd605850b15337de63a0163b31123182dae245024f9a0ca714effab8a"}
Jul 06 23:49:21.690510 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.690461    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s8jnd" podStartSLOduration=2.408555743 podStartE2EDuration="12.690445027s" totalImagesPullingTime="10.281889284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:49:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:21.689938685 +0000 UTC m=+62.755270665" watchObservedRunningTime="2026-07-06 23:49:21.690445027 +0000 UTC m=+62.755777001"
Jul 06 23:49:21.742795 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7bfa3930-3794-45ea-94ba-3b019691282f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.742795 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-sys\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742818    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-tls\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742865    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-root\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742906    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-sys\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bfa3930-3794-45ea-94ba-3b019691282f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.743011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742976    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-root\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743011 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.742980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.743215 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qthng\" (UniqueName: \"kubernetes.io/projected/7bfa3930-3794-45ea-94ba-3b019691282f-kube-api-access-qthng\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.743215 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743065    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-textfile\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743215 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-accelerators-collector-config\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743215 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.743215 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-metrics-client-ca\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.743473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743210    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7bfa3930-3794-45ea-94ba-3b019691282f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwkpj\" (UniqueName: \"kubernetes.io/projected/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-kube-api-access-zwkpj\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-wtmp\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743736    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.743785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:21.744118    2564 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:21.744190    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-tls podName:7bfa3930-3794-45ea-94ba-3b019691282f nodeName:}" failed. No retries permitted until 2026-07-06 23:49:22.244171289 +0000 UTC m=+63.309503251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-f6t6n" (UID: "7bfa3930-3794-45ea-94ba-3b019691282f") : secret "kube-state-metrics-tls" not found
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.744219    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-wtmp\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.744666 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.744584    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-textfile\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.745155 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.745135    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-metrics-client-ca\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.745213 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.745178    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-accelerators-collector-config\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.745303 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.745284    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.745515 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.745498    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bfa3930-3794-45ea-94ba-3b019691282f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.746448 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.746408    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.746545 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.746501    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-tls\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.747066 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.747045    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.753177 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.753153    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwkpj\" (UniqueName: \"kubernetes.io/projected/cc8f0e2a-fd77-49b1-b8e3-3382a6671f62-kube-api-access-zwkpj\") pod \"node-exporter-fsmg9\" (UID: \"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62\") " pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.754061 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.754046    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qthng\" (UniqueName: \"kubernetes.io/projected/7bfa3930-3794-45ea-94ba-3b019691282f-kube-api-access-qthng\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:21.900343 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.900237    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fsmg9"
Jul 06 23:49:21.909841 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:21.909808    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8f0e2a_fd77_49b1_b8e3_3382a6671f62.slice/crio-89fb9742c9408ed43a8d747d8cd580aee3f1a2f2c242a68b4d1e5732f6659aff WatchSource:0}: Error finding container 89fb9742c9408ed43a8d747d8cd580aee3f1a2f2c242a68b4d1e5732f6659aff: Status 404 returned error can't find the container with id 89fb9742c9408ed43a8d747d8cd580aee3f1a2f2c242a68b4d1e5732f6659aff
Jul 06 23:49:21.937497 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.937459    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:21.937652 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.937520    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:21.938736 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.938708    2564 patch_prober.go:28] interesting pod/console-6497d74f5d-gn9hd container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 06 23:49:21.938861 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:21.938755    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6497d74f5d-gn9hd" podUID="52106a2b-62a9-4ada-b374-d0f70bbd7f7d" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 06 23:49:22.247878 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.247842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:22.250417 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.250392    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bfa3930-3794-45ea-94ba-3b019691282f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f6t6n\" (UID: \"7bfa3930-3794-45ea-94ba-3b019691282f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:22.492961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.492923    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"
Jul 06 23:49:22.644710 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.644515    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n"]
Jul 06 23:49:22.647558 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:22.647525    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfa3930_3794_45ea_94ba_3b019691282f.slice/crio-c6f4a191292793b7aaac7053cb4e5580427df286d84f7f004bb3fcdeefc6bc38 WatchSource:0}: Error finding container c6f4a191292793b7aaac7053cb4e5580427df286d84f7f004bb3fcdeefc6bc38: Status 404 returned error can't find the container with id c6f4a191292793b7aaac7053cb4e5580427df286d84f7f004bb3fcdeefc6bc38
Jul 06 23:49:22.669989 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.669955    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:49:22.673475 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.673448    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.674267 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.674237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fsmg9" event={"ID":"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62","Type":"ContainerStarted","Data":"89fb9742c9408ed43a8d747d8cd580aee3f1a2f2c242a68b4d1e5732f6659aff"}
Jul 06 23:49:22.676903 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.676297    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n" event={"ID":"7bfa3930-3794-45ea-94ba-3b019691282f","Type":"ContainerStarted","Data":"c6f4a191292793b7aaac7053cb4e5580427df286d84f7f004bb3fcdeefc6bc38"}
Jul 06 23:49:22.677178 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.677150    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 23:49:22.677375 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.677354    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 23:49:22.677574 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.677555    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 23:49:22.677817 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.677801    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 23:49:22.678177 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.678158    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-99nqs\""
Jul 06 23:49:22.678783 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.678766    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 23:49:22.678993 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.678973    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 23:49:22.679190 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.679174    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 23:49:22.679375 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.679361    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 23:49:22.680016 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.679999    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 23:49:22.689465 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.689411    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:49:22.752911 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.752827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753057 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.752968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753057 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-web-config\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753170 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753062    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753170 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-volume\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753276 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753276 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753210    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753276 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753276 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753471 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcxc9\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-kube-api-access-rcxc9\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753471 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753309    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753471 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753330    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-out\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.753471 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.753361    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854468 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-web-config\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-volume\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxc9\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-kube-api-access-rcxc9\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-out\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.854961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.855246 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.854990    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.857143 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:22.855560    2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 06 23:49:22.857143 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:22.855630    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls podName:7f56bb2d-96c8-4ccc-a2a3-8df2d243b311 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:23.355607945 +0000 UTC m=+64.420939917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311") : secret "alertmanager-main-tls" not found
Jul 06 23:49:22.857143 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.856791    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.857809 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.857783    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-web-config\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.857905 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.857792    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.857905 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:22.857897    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle podName:7f56bb2d-96c8-4ccc-a2a3-8df2d243b311 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:23.357879853 +0000 UTC m=+64.423211816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311") : configmap references non-existent config key: ca-bundle.crt
Jul 06 23:49:22.858026 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.857961    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-volume\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.858026 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.857991    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.858026 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.858017    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.858338 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.858219    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.860141 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.860117    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-out\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.860330 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.860312    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:22.864853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:22.864833    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcxc9\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-kube-api-access-rcxc9\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:23.359615 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:23.359534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:23.359615 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:23.359572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:23.360375 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:23.360356    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:23.362040 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:23.362015    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:23.587008 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:23.586972    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:49:23.681745 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:23.681660    2564 generic.go:356] "Generic (PLEG): container finished" podID="cc8f0e2a-fd77-49b1-b8e3-3382a6671f62" containerID="a6c9808f84744633f8d2654470df211344e3b6c66b16e7f136651754fa8c25e4" exitCode=0
Jul 06 23:49:23.681745 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:23.681715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fsmg9" event={"ID":"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62","Type":"ContainerDied","Data":"a6c9808f84744633f8d2654470df211344e3b6c66b16e7f136651754fa8c25e4"}
Jul 06 23:49:24.031195 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.031163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:49:24.036505 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:24.036471    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f56bb2d_96c8_4ccc_a2a3_8df2d243b311.slice/crio-03c4d3ca1f1dcd43df7af6e045d380c8336f2c99f659427ac41b2df1a1b0f66b WatchSource:0}: Error finding container 03c4d3ca1f1dcd43df7af6e045d380c8336f2c99f659427ac41b2df1a1b0f66b: Status 404 returned error can't find the container with id 03c4d3ca1f1dcd43df7af6e045d380c8336f2c99f659427ac41b2df1a1b0f66b
Jul 06 23:49:24.688495 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.688452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n" event={"ID":"7bfa3930-3794-45ea-94ba-3b019691282f","Type":"ContainerStarted","Data":"78c6f64c3d0b4c69a583c9f6bd94e9f9afe44dd590c0567bb67e8197c9080286"}
Jul 06 23:49:24.688934 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.688527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n" event={"ID":"7bfa3930-3794-45ea-94ba-3b019691282f","Type":"ContainerStarted","Data":"8720364d3d9cbd1e79e615693a4f43232ba4bbcd3170cd57eff4e8034a9a2da8"}
Jul 06 23:49:24.688934 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.688546    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n" event={"ID":"7bfa3930-3794-45ea-94ba-3b019691282f","Type":"ContainerStarted","Data":"a6cb4a7c5f84eb388c1937a9873a4b9483aff1c4c7588215690a782dd9634c7d"}
Jul 06 23:49:24.691038 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.691007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fsmg9" event={"ID":"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62","Type":"ContainerStarted","Data":"9de265d1ff66cc799721dd38bc2df52931d90bf64350a328c4b8516483cafb36"}
Jul 06 23:49:24.691155 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.691060    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fsmg9" event={"ID":"cc8f0e2a-fd77-49b1-b8e3-3382a6671f62","Type":"ContainerStarted","Data":"2cf6e5a52a1c04eb2af999843aa197a125254cf668ea9f806fbbdedc624e7c6c"}
Jul 06 23:49:24.692257 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.692235    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerStarted","Data":"03c4d3ca1f1dcd43df7af6e045d380c8336f2c99f659427ac41b2df1a1b0f66b"}
Jul 06 23:49:24.709514 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.709460    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f6t6n" podStartSLOduration=2.431663067 podStartE2EDuration="3.709438357s" totalImagesPullingTime="1.27777529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:24.709136608 +0000 UTC m=+65.774468584" watchObservedRunningTime="2026-07-06 23:49:24.709438357 +0000 UTC m=+65.774770334"
Jul 06 23:49:24.735103 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:24.735052    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fsmg9" podStartSLOduration=2.522773898 podStartE2EDuration="3.73503613s" totalImagesPullingTime="1.142293791s" totalInitContainerRuntime="69.968441ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:24.734601744 +0000 UTC m=+65.799933725" watchObservedRunningTime="2026-07-06 23:49:24.73503613 +0000 UTC m=+65.800368106"
Jul 06 23:49:25.079362 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.079325    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:25.079577 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.079392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:25.082218 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.082194    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:49:25.082319 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.082236    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:49:25.093187 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.093162    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4c5715-f342-45cd-a938-c93cf23382b5-original-pull-secret\") pod \"global-pull-secret-syncer-s4bnc\" (UID: \"9c4c5715-f342-45cd-a938-c93cf23382b5\") " pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:25.093345 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.093324    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94f2f896-2166-460e-9cec-8cb2e977fa92-metrics-certs\") pod \"network-metrics-daemon-56p5q\" (UID: \"94f2f896-2166-460e-9cec-8cb2e977fa92\") " pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:25.100865 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.100838    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s4bnc"
Jul 06 23:49:25.111488 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.111469    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjdfk\""
Jul 06 23:49:25.119674 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.119493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-56p5q"
Jul 06 23:49:25.180323 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.179892    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:25.182856 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.182821    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:49:25.193475 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.193354    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:49:25.204583 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.204526    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdv8g\" (UniqueName: \"kubernetes.io/projected/0b21f861-6631-4376-8b6e-1980034ea780-kube-api-access-bdv8g\") pod \"network-check-target-gl4f5\" (UID: \"0b21f861-6631-4376-8b6e-1980034ea780\") " pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:25.247976 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.247930    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s4bnc"]
Jul 06 23:49:25.251656 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:25.251587    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4c5715_f342_45cd_a938_c93cf23382b5.slice/crio-caaa4de99abba47811652f0cfac2752a7ad303eb3cf7494bd27e928d324c4738 WatchSource:0}: Error finding container caaa4de99abba47811652f0cfac2752a7ad303eb3cf7494bd27e928d324c4738: Status 404 returned error can't find the container with id caaa4de99abba47811652f0cfac2752a7ad303eb3cf7494bd27e928d324c4738
Jul 06 23:49:25.265186 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.265163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-56p5q"]
Jul 06 23:49:25.316298 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:25.316265    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f2f896_2166_460e_9cec_8cb2e977fa92.slice/crio-573e3916b042107801e99f0ba91dfdb91321eb5c56b2bea468311153d264a8d4 WatchSource:0}: Error finding container 573e3916b042107801e99f0ba91dfdb91321eb5c56b2bea468311153d264a8d4: Status 404 returned error can't find the container with id 573e3916b042107801e99f0ba91dfdb91321eb5c56b2bea468311153d264a8d4
Jul 06 23:49:25.501138 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.501111    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m28zf\""
Jul 06 23:49:25.509238 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.509212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:25.632297 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.632267    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gl4f5"]
Jul 06 23:49:25.635970 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:25.635947    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b21f861_6631_4376_8b6e_1980034ea780.slice/crio-9c576531ea4072908c07755de0c02675cecda1288f10afc8187b176672ca3035 WatchSource:0}: Error finding container 9c576531ea4072908c07755de0c02675cecda1288f10afc8187b176672ca3035: Status 404 returned error can't find the container with id 9c576531ea4072908c07755de0c02675cecda1288f10afc8187b176672ca3035
Jul 06 23:49:25.697497 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.697458    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-56p5q" event={"ID":"94f2f896-2166-460e-9cec-8cb2e977fa92","Type":"ContainerStarted","Data":"573e3916b042107801e99f0ba91dfdb91321eb5c56b2bea468311153d264a8d4"}
Jul 06 23:49:25.699361 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.699335    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerID="f03a33893ed9da384df96ce0e6aa6000bf80e191823a025f6cac9baa22999444" exitCode=0
Jul 06 23:49:25.699592 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.699391    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"f03a33893ed9da384df96ce0e6aa6000bf80e191823a025f6cac9baa22999444"}
Jul 06 23:49:25.700963 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.700940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gl4f5" event={"ID":"0b21f861-6631-4376-8b6e-1980034ea780","Type":"ContainerStarted","Data":"9c576531ea4072908c07755de0c02675cecda1288f10afc8187b176672ca3035"}
Jul 06 23:49:25.702414 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:25.702391    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s4bnc" event={"ID":"9c4c5715-f342-45cd-a938-c93cf23382b5","Type":"ContainerStarted","Data":"caaa4de99abba47811652f0cfac2752a7ad303eb3cf7494bd27e928d324c4738"}
Jul 06 23:49:26.001436 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.001380    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-77b456fbdb-t5jlj"]
Jul 06 23:49:26.004906 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.004885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.008379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.007790    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 06 23:49:26.008379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.008025    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 06 23:49:26.008379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.008239    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6sodgk84e476l\""
Jul 06 23:49:26.008819 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.008782    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 23:49:26.008913 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.008820    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sbpn9\""
Jul 06 23:49:26.009031 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.009016    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 06 23:49:26.017648 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.017589    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77b456fbdb-t5jlj"]
Jul 06 23:49:26.089776 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.089721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-client-ca-bundle\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.089961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.089810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-secret-metrics-server-client-certs\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.089961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.089846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flx52\" (UniqueName: \"kubernetes.io/projected/ee01e8f0-4a6c-4013-951c-f89cffa559dd-kube-api-access-flx52\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.089961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.089873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-secret-metrics-server-tls\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.089961 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.089909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee01e8f0-4a6c-4013-951c-f89cffa559dd-metrics-server-audit-profiles\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.090207 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.089963    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee01e8f0-4a6c-4013-951c-f89cffa559dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.090207 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.090015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee01e8f0-4a6c-4013-951c-f89cffa559dd-audit-log\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.190779 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.190737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-client-ca-bundle\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.190966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.190826    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-secret-metrics-server-client-certs\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.190966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.190873    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flx52\" (UniqueName: \"kubernetes.io/projected/ee01e8f0-4a6c-4013-951c-f89cffa559dd-kube-api-access-flx52\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.190966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.190900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-secret-metrics-server-tls\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.190966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.190929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee01e8f0-4a6c-4013-951c-f89cffa559dd-metrics-server-audit-profiles\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.191139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.190977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee01e8f0-4a6c-4013-951c-f89cffa559dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.191139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.191002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee01e8f0-4a6c-4013-951c-f89cffa559dd-audit-log\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.191439 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.191408    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee01e8f0-4a6c-4013-951c-f89cffa559dd-audit-log\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.192578 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.192552    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee01e8f0-4a6c-4013-951c-f89cffa559dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.192853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.192819    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee01e8f0-4a6c-4013-951c-f89cffa559dd-metrics-server-audit-profiles\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.194500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.194455    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-client-ca-bundle\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.194601 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.194533    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-secret-metrics-server-tls\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.195047 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.195021    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee01e8f0-4a6c-4013-951c-f89cffa559dd-secret-metrics-server-client-certs\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.200879 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.200858    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flx52\" (UniqueName: \"kubernetes.io/projected/ee01e8f0-4a6c-4013-951c-f89cffa559dd-kube-api-access-flx52\") pod \"metrics-server-77b456fbdb-t5jlj\" (UID: \"ee01e8f0-4a6c-4013-951c-f89cffa559dd\") " pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.320718 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.320672    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:26.399962 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.397254    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6497d74f5d-gn9hd"]
Jul 06 23:49:26.433870 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.432470    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-689645f56f-bj2lp"]
Jul 06 23:49:26.439458 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.439405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.443243 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.443181    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689645f56f-bj2lp"]
Jul 06 23:49:26.449046 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.448930    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 23:49:26.513211 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.513057    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77b456fbdb-t5jlj"]
Jul 06 23:49:26.522938 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:26.522907    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee01e8f0_4a6c_4013_951c_f89cffa559dd.slice/crio-00efff153d4de7d7e15cde41cb2513f09ea4a4eb3a55fd91430820125156e351 WatchSource:0}: Error finding container 00efff153d4de7d7e15cde41cb2513f09ea4a4eb3a55fd91430820125156e351: Status 404 returned error can't find the container with id 00efff153d4de7d7e15cde41cb2513f09ea4a4eb3a55fd91430820125156e351
Jul 06 23:49:26.596814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.596511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-oauth-config\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.596814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.596573    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-trusted-ca-bundle\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.596814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.596622    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-service-ca\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.596814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.596658    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-config\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.596814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.596689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-serving-cert\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.596814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.596726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-oauth-serving-cert\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.596814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.596752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czpz8\" (UniqueName: \"kubernetes.io/projected/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-kube-api-access-czpz8\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.698256 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.698222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-trusted-ca-bundle\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.698712 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.698268    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-service-ca\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.698712 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.698309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-config\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.698712 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.698343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-serving-cert\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.698712 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.698378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-oauth-serving-cert\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.698712 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.698401    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czpz8\" (UniqueName: \"kubernetes.io/projected/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-kube-api-access-czpz8\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.698712 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.698472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-oauth-config\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.699269 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.699245    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-config\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.699406 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.699384    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-oauth-serving-cert\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.700237 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.700071    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-service-ca\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.700237 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.700114    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-trusted-ca-bundle\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.702115 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.702090    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-oauth-config\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.704285 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.704242    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-serving-cert\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.708660 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.708638    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czpz8\" (UniqueName: \"kubernetes.io/projected/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-kube-api-access-czpz8\") pod \"console-689645f56f-bj2lp\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") " pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.712442 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.712140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-56p5q" event={"ID":"94f2f896-2166-460e-9cec-8cb2e977fa92","Type":"ContainerStarted","Data":"b993900439cd10ac2ed871d567ac9c02c7584fb22d526ed2136fba6913d755cf"}
Jul 06 23:49:26.712442 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.712199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-56p5q" event={"ID":"94f2f896-2166-460e-9cec-8cb2e977fa92","Type":"ContainerStarted","Data":"12c5245d13ed73f4de567a7a664492952ad560dfbfebd05b1316f53e618739de"}
Jul 06 23:49:26.713828 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.713786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj" event={"ID":"ee01e8f0-4a6c-4013-951c-f89cffa559dd","Type":"ContainerStarted","Data":"00efff153d4de7d7e15cde41cb2513f09ea4a4eb3a55fd91430820125156e351"}
Jul 06 23:49:26.730232 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.730183    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-56p5q" podStartSLOduration=66.78073771 podStartE2EDuration="1m7.730165581s" totalImagesPullingTime="949.427871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:26.729723856 +0000 UTC m=+67.795055834" watchObservedRunningTime="2026-07-06 23:49:26.730165581 +0000 UTC m=+67.795497559"
Jul 06 23:49:26.762631 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.762409    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:26.979706 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:26.979602    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689645f56f-bj2lp"]
Jul 06 23:49:26.988198 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:26.988160    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e63bf0_0f87_49d8_bfd8_34bbad9dc866.slice/crio-4a44ae281d1fc0aa191e3e20a94fd07c168d7b9f99f51820fa7cea05b38da370 WatchSource:0}: Error finding container 4a44ae281d1fc0aa191e3e20a94fd07c168d7b9f99f51820fa7cea05b38da370: Status 404 returned error can't find the container with id 4a44ae281d1fc0aa191e3e20a94fd07c168d7b9f99f51820fa7cea05b38da370
Jul 06 23:49:27.653844 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.653357    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nl57w"
Jul 06 23:49:27.736848 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.736521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689645f56f-bj2lp" event={"ID":"23e63bf0-0f87-49d8-bfd8-34bbad9dc866","Type":"ContainerStarted","Data":"bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4"}
Jul 06 23:49:27.736848 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.736602    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689645f56f-bj2lp" event={"ID":"23e63bf0-0f87-49d8-bfd8-34bbad9dc866","Type":"ContainerStarted","Data":"4a44ae281d1fc0aa191e3e20a94fd07c168d7b9f99f51820fa7cea05b38da370"}
Jul 06 23:49:27.757116 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.756984    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-689645f56f-bj2lp" podStartSLOduration=1.7569652 podStartE2EDuration="1.7569652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:27.756178178 +0000 UTC m=+68.821510156" watchObservedRunningTime="2026-07-06 23:49:27.7569652 +0000 UTC m=+68.822297179"
Jul 06 23:49:27.856901 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.856034    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:49:27.890613 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.890510    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:49:27.890783 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.890647    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.893829    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.894227    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.894479    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.894844    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-g4xph\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.895032    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.895192    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.895454    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.895646    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.895832    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4mcl4t0upgubi\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.896541    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.896762    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.896992    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.897171    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.897388    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 23:49:27.898354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.897571    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 23:49:27.901829 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:27.900088    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 23:49:28.009264 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009238    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009305    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009331    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009447    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009470    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9s8b\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-kube-api-access-v9s8b\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009701    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009824    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.009971 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.009853    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111310 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111454    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111521 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111510    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111565    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111590    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111642    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111739    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111767    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.111832 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.112296 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9s8b\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-kube-api-access-v9s8b\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.112296 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111890    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.112296 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.112296 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.111955    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.116853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.112774    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.116853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.113240    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.116853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.113692    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.116853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.115299    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.116853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.115486    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.116853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.116235    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.116853 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.116623    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.124561 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.124507    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.126254 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.125856    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.126254 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.126174    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.126254 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.126183    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.126254 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.126238    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.126731 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.126669    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.127374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.126858    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.127374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.126874    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.127374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.126951    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.127374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.127051    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.127374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.127325    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.129903 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.129868    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9s8b\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-kube-api-access-v9s8b\") pod \"prometheus-k8s-0\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:28.209032 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:28.208949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:31.194384 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.194289    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:49:31.203800 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:31.203770    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6b12b1_e3ce_4d72_89ef_fa82563f7b77.slice/crio-977972ba5cd70217d9568bad9fac529a6f4c42e23a31b95585e00e82b25817da WatchSource:0}: Error finding container 977972ba5cd70217d9568bad9fac529a6f4c42e23a31b95585e00e82b25817da: Status 404 returned error can't find the container with id 977972ba5cd70217d9568bad9fac529a6f4c42e23a31b95585e00e82b25817da
Jul 06 23:49:31.748721 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.748686    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj" event={"ID":"ee01e8f0-4a6c-4013-951c-f89cffa559dd","Type":"ContainerStarted","Data":"9bea8bcb1248516a87145632324875f022d28165c43dd963401ee54baf326bea"}
Jul 06 23:49:31.751436 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.751397    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerStarted","Data":"3bbfcc67bd371f57a4f9661f7edc4ff2f13891024ed6179a6b6e76f41a25fbb7"}
Jul 06 23:49:31.751552 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.751489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerStarted","Data":"25c7a31fae8f0ea82cbf4c76d2bca606812d51c688c8cc75a01884eef8d1155a"}
Jul 06 23:49:31.751552 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.751530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerStarted","Data":"e20007f48edded56ab85ffaed092544b2f7a944557abba0803e33c32d64f2958"}
Jul 06 23:49:31.751660 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.751566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerStarted","Data":"83bbd3d23941d268742d75844605c499b5cb05d1f7c2818426020fd67a02df34"}
Jul 06 23:49:31.751660 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.751606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerStarted","Data":"eb892f79e5c7ba11217929d018d981940e6c002b9cba6404459e45587af015ff"}
Jul 06 23:49:31.752771 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.752749    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05" exitCode=0
Jul 06 23:49:31.752883 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.752810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"}
Jul 06 23:49:31.752941 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.752882    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerStarted","Data":"977972ba5cd70217d9568bad9fac529a6f4c42e23a31b95585e00e82b25817da"}
Jul 06 23:49:31.754230 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.754210    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gl4f5" event={"ID":"0b21f861-6631-4376-8b6e-1980034ea780","Type":"ContainerStarted","Data":"04a443f38aa315be6efabf5e99094fbcdb0ac67268e417077d725aad05e90e73"}
Jul 06 23:49:31.754354 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.754341    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:49:31.755684 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.755663    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s4bnc" event={"ID":"9c4c5715-f342-45cd-a938-c93cf23382b5","Type":"ContainerStarted","Data":"fc1570ca8b3c1a3eb871a20fbea3d0b4e261883b1f733a4982b3f7ff937691b0"}
Jul 06 23:49:31.768414 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.768360    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj" podStartSLOduration=2.285369575 podStartE2EDuration="6.768346247s" totalImagesPullingTime="4.482976672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:31.767661258 +0000 UTC m=+72.832993237" watchObservedRunningTime="2026-07-06 23:49:31.768346247 +0000 UTC m=+72.833678223"
Jul 06 23:49:31.785397 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.785344    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gl4f5" podStartSLOduration=67.337717768 podStartE2EDuration="1m12.785328259s" totalImagesPullingTime="5.447610491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:31.785010853 +0000 UTC m=+72.850342823" watchObservedRunningTime="2026-07-06 23:49:31.785328259 +0000 UTC m=+72.850660237"
Jul 06 23:49:31.802205 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:31.802159    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s4bnc" podStartSLOduration=66.989192313 podStartE2EDuration="1m12.802146903s" totalImagesPullingTime="5.81295459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:31.801635649 +0000 UTC m=+72.866967626" watchObservedRunningTime="2026-07-06 23:49:31.802146903 +0000 UTC m=+72.867478879"
Jul 06 23:49:32.764211 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:32.764173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerStarted","Data":"5037369be723a2ae133b4b0f68c2feede9c1c02974874ed475de045270c5cc59"}
Jul 06 23:49:32.795723 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:32.795670    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.302195828 podStartE2EDuration="10.795651207s" totalImagesPullingTime="7.474801659s" totalInitContainerRuntime="18.65372ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:32.793160448 +0000 UTC m=+73.858492429" watchObservedRunningTime="2026-07-06 23:49:32.795651207 +0000 UTC m=+73.860983186"
Jul 06 23:49:35.781110 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:35.781072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerStarted","Data":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"}
Jul 06 23:49:35.781555 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:35.781180    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerStarted","Data":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"}
Jul 06 23:49:36.765617 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:36.765251    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:36.765617 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:36.765328    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:36.771814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:36.771780    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:36.790786 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:36.790577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:49:37.792886 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:37.792853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerStarted","Data":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"}
Jul 06 23:49:37.793268 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:37.792916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerStarted","Data":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"}
Jul 06 23:49:37.793268 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:37.792940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerStarted","Data":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"}
Jul 06 23:49:37.793268 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:37.793008    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerStarted","Data":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"}
Jul 06 23:49:38.210232 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.210034    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:49:38.361394 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.361335    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.221646225 podStartE2EDuration="11.361315892s" totalImagesPullingTime="5.11807494s" totalInitContainerRuntime="21.594727ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:37.823858877 +0000 UTC m=+78.889190857" watchObservedRunningTime="2026-07-06 23:49:38.361315892 +0000 UTC m=+79.426647869"
Jul 06 23:49:38.361892 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.361869    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76b4b8687b-5cqqt"]
Jul 06 23:49:38.365010 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.364996    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.377814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.377787    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b4b8687b-5cqqt"]
Jul 06 23:49:38.420890 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.420858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-trusted-ca-bundle\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.420890 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.420893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-oauth-config\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.421081 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.420915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzqgz\" (UniqueName: \"kubernetes.io/projected/07f0a593-ecd0-4c29-8269-c478971ca94b-kube-api-access-bzqgz\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.421081 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.420998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-oauth-serving-cert\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.421081 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.421043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-service-ca\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.421081 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.421065    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-console-config\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.421201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.421089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-serving-cert\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.521706 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.521671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-trusted-ca-bundle\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.521706 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.521707    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-oauth-config\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.521906 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.521734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzqgz\" (UniqueName: \"kubernetes.io/projected/07f0a593-ecd0-4c29-8269-c478971ca94b-kube-api-access-bzqgz\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.521906 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.521775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-oauth-serving-cert\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.521906 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.521826    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-service-ca\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.521906 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.521855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-console-config\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.521906 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.521884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-serving-cert\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.522619 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.522593    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-service-ca\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.522707 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.522640    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-trusted-ca-bundle\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.522819 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.522796    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-oauth-serving-cert\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.522964 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.522943    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-console-config\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.524389 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.524357    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-oauth-config\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.524527 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.524509    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-serving-cert\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.532146 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.532122    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzqgz\" (UniqueName: \"kubernetes.io/projected/07f0a593-ecd0-4c29-8269-c478971ca94b-kube-api-access-bzqgz\") pod \"console-76b4b8687b-5cqqt\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") " pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.674039 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.674005    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:38.818685 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:38.818662    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b4b8687b-5cqqt"]
Jul 06 23:49:38.821106 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:49:38.821084    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f0a593_ecd0_4c29_8269_c478971ca94b.slice/crio-ec830534d6a48ddff596de962860354bbbf26ab0eac3dfa965dee1c84c7dfe79 WatchSource:0}: Error finding container ec830534d6a48ddff596de962860354bbbf26ab0eac3dfa965dee1c84c7dfe79: Status 404 returned error can't find the container with id ec830534d6a48ddff596de962860354bbbf26ab0eac3dfa965dee1c84c7dfe79
Jul 06 23:49:39.802455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:39.802296    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b4b8687b-5cqqt" event={"ID":"07f0a593-ecd0-4c29-8269-c478971ca94b","Type":"ContainerStarted","Data":"7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf"}
Jul 06 23:49:39.802455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:39.802390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b4b8687b-5cqqt" event={"ID":"07f0a593-ecd0-4c29-8269-c478971ca94b","Type":"ContainerStarted","Data":"ec830534d6a48ddff596de962860354bbbf26ab0eac3dfa965dee1c84c7dfe79"}
Jul 06 23:49:39.822563 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:39.822503    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76b4b8687b-5cqqt" podStartSLOduration=1.822486007 podStartE2EDuration="1.822486007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:39.821356887 +0000 UTC m=+80.886688894" watchObservedRunningTime="2026-07-06 23:49:39.822486007 +0000 UTC m=+80.887817983"
Jul 06 23:49:46.321382 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:46.321351    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:46.321908 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:46.321443    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:49:48.674231 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:48.674188    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:48.674633 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:48.674251    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:48.678992 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:48.678970    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:48.843714 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:48.843686    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:49:48.904174 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:48.904139    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689645f56f-bj2lp"]
Jul 06 23:49:51.433341 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.433276    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6497d74f5d-gn9hd" podUID="52106a2b-62a9-4ada-b374-d0f70bbd7f7d" containerName="console" containerID="cri-o://defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736" gracePeriod=15
Jul 06 23:49:51.681536 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.681508    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:51.740014 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.739964    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-serving-cert\") pod \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") "
Jul 06 23:49:51.740204 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740032    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-config\") pod \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") "
Jul 06 23:49:51.740204 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740087    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-oauth-serving-cert\") pod \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") "
Jul 06 23:49:51.740204 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740123    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-service-ca\") pod \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") "
Jul 06 23:49:51.740204 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740150    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-oauth-config\") pod \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") "
Jul 06 23:49:51.740204 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fngk4\" (UniqueName: \"kubernetes.io/projected/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-kube-api-access-fngk4\") pod \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\" (UID: \"52106a2b-62a9-4ada-b374-d0f70bbd7f7d\") "
Jul 06 23:49:51.740483 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740439    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-config" (OuterVolumeSpecName: "console-config") pod "52106a2b-62a9-4ada-b374-d0f70bbd7f7d" (UID: "52106a2b-62a9-4ada-b374-d0f70bbd7f7d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:49:51.740483 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740444    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "52106a2b-62a9-4ada-b374-d0f70bbd7f7d" (UID: "52106a2b-62a9-4ada-b374-d0f70bbd7f7d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:49:51.740594 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.740561    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-service-ca" (OuterVolumeSpecName: "service-ca") pod "52106a2b-62a9-4ada-b374-d0f70bbd7f7d" (UID: "52106a2b-62a9-4ada-b374-d0f70bbd7f7d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:49:51.742678 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.742643    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "52106a2b-62a9-4ada-b374-d0f70bbd7f7d" (UID: "52106a2b-62a9-4ada-b374-d0f70bbd7f7d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:49:51.742678 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.742670    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-kube-api-access-fngk4" (OuterVolumeSpecName: "kube-api-access-fngk4") pod "52106a2b-62a9-4ada-b374-d0f70bbd7f7d" (UID: "52106a2b-62a9-4ada-b374-d0f70bbd7f7d"). InnerVolumeSpecName "kube-api-access-fngk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:49:51.742834 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.742698    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "52106a2b-62a9-4ada-b374-d0f70bbd7f7d" (UID: "52106a2b-62a9-4ada-b374-d0f70bbd7f7d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:49:51.841229 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.841189    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-serving-cert\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:49:51.841229 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.841224    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:49:51.841229 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.841233    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-oauth-serving-cert\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:49:51.841229 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.841242    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:49:51.841535 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.841250    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-console-oauth-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:49:51.841535 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.841259    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fngk4\" (UniqueName: \"kubernetes.io/projected/52106a2b-62a9-4ada-b374-d0f70bbd7f7d-kube-api-access-fngk4\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:49:51.852828 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.852794    2564 generic.go:356] "Generic (PLEG): container finished" podID="52106a2b-62a9-4ada-b374-d0f70bbd7f7d" containerID="defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736" exitCode=0
Jul 06 23:49:51.852982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.852843    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6497d74f5d-gn9hd"
Jul 06 23:49:51.852982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.852850    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6497d74f5d-gn9hd" event={"ID":"52106a2b-62a9-4ada-b374-d0f70bbd7f7d","Type":"ContainerDied","Data":"defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736"}
Jul 06 23:49:51.852982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.852889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6497d74f5d-gn9hd" event={"ID":"52106a2b-62a9-4ada-b374-d0f70bbd7f7d","Type":"ContainerDied","Data":"c779512e2140378e78923cbe583a13921e70a2ca53ebbb024344098ca36412ce"}
Jul 06 23:49:51.852982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.852910    2564 scope.go:117] "RemoveContainer" containerID="defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736"
Jul 06 23:49:51.861830 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.861811    2564 scope.go:117] "RemoveContainer" containerID="defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736"
Jul 06 23:49:51.862151 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:49:51.862126    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736\": container with ID starting with defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736 not found: ID does not exist" containerID="defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736"
Jul 06 23:49:51.862253 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.862158    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736"} err="failed to get container status \"defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736\": rpc error: code = NotFound desc = could not find container \"defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736\": container with ID starting with defbacd63dfaf22df71e1c522da0afdd6c2dd49e5ebb926a5fe089ed0f8bc736 not found: ID does not exist"
Jul 06 23:49:51.875156 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.875123    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6497d74f5d-gn9hd"]
Jul 06 23:49:51.879209 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:51.879181    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6497d74f5d-gn9hd"]
Jul 06 23:49:53.390075 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:49:53.390035    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52106a2b-62a9-4ada-b374-d0f70bbd7f7d" path="/var/lib/kubelet/pods/52106a2b-62a9-4ada-b374-d0f70bbd7f7d/volumes"
Jul 06 23:50:02.766737 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:02.766703    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gl4f5"
Jul 06 23:50:06.326089 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:06.326059    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:50:06.330000 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:06.329973    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-77b456fbdb-t5jlj"
Jul 06 23:50:13.925398 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:13.925349    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-689645f56f-bj2lp" podUID="23e63bf0-0f87-49d8-bfd8-34bbad9dc866" containerName="console" containerID="cri-o://bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4" gracePeriod=15
Jul 06 23:50:14.175594 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.175531    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:50:14.337409 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337367    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-config\") pod \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") "
Jul 06 23:50:14.337623 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337475    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-trusted-ca-bundle\") pod \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") "
Jul 06 23:50:14.337623 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337501    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-oauth-config\") pod \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") "
Jul 06 23:50:14.337623 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337519    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-oauth-serving-cert\") pod \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") "
Jul 06 23:50:14.337623 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337537    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czpz8\" (UniqueName: \"kubernetes.io/projected/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-kube-api-access-czpz8\") pod \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") "
Jul 06 23:50:14.337836 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337673    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-service-ca\") pod \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") "
Jul 06 23:50:14.337836 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337715    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-serving-cert\") pod \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\" (UID: \"23e63bf0-0f87-49d8-bfd8-34bbad9dc866\") "
Jul 06 23:50:14.337949 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337879    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-config" (OuterVolumeSpecName: "console-config") pod "23e63bf0-0f87-49d8-bfd8-34bbad9dc866" (UID: "23e63bf0-0f87-49d8-bfd8-34bbad9dc866"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:14.338002 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.337972    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "23e63bf0-0f87-49d8-bfd8-34bbad9dc866" (UID: "23e63bf0-0f87-49d8-bfd8-34bbad9dc866"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:14.338051 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.338004    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:14.338051 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.338003    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "23e63bf0-0f87-49d8-bfd8-34bbad9dc866" (UID: "23e63bf0-0f87-49d8-bfd8-34bbad9dc866"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:14.338051 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.338015    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-service-ca" (OuterVolumeSpecName: "service-ca") pod "23e63bf0-0f87-49d8-bfd8-34bbad9dc866" (UID: "23e63bf0-0f87-49d8-bfd8-34bbad9dc866"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:14.339968 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.339939    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-kube-api-access-czpz8" (OuterVolumeSpecName: "kube-api-access-czpz8") pod "23e63bf0-0f87-49d8-bfd8-34bbad9dc866" (UID: "23e63bf0-0f87-49d8-bfd8-34bbad9dc866"). InnerVolumeSpecName "kube-api-access-czpz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:50:14.339968 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.339954    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "23e63bf0-0f87-49d8-bfd8-34bbad9dc866" (UID: "23e63bf0-0f87-49d8-bfd8-34bbad9dc866"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:14.340155 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.340011    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "23e63bf0-0f87-49d8-bfd8-34bbad9dc866" (UID: "23e63bf0-0f87-49d8-bfd8-34bbad9dc866"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:14.439473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.439365    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:14.439473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.439400    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-serving-cert\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:14.439473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.439411    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-trusted-ca-bundle\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:14.439473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.439446    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-console-oauth-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:14.439473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.439455    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-oauth-serving-cert\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:14.439473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.439465    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czpz8\" (UniqueName: \"kubernetes.io/projected/23e63bf0-0f87-49d8-bfd8-34bbad9dc866-kube-api-access-czpz8\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:14.952072 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.952034    2564 generic.go:356] "Generic (PLEG): container finished" podID="23e63bf0-0f87-49d8-bfd8-34bbad9dc866" containerID="bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4" exitCode=0
Jul 06 23:50:14.952500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.952084    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689645f56f-bj2lp"
Jul 06 23:50:14.952500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.952115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689645f56f-bj2lp" event={"ID":"23e63bf0-0f87-49d8-bfd8-34bbad9dc866","Type":"ContainerDied","Data":"bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4"}
Jul 06 23:50:14.952500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.952166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689645f56f-bj2lp" event={"ID":"23e63bf0-0f87-49d8-bfd8-34bbad9dc866","Type":"ContainerDied","Data":"4a44ae281d1fc0aa191e3e20a94fd07c168d7b9f99f51820fa7cea05b38da370"}
Jul 06 23:50:14.952500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.952187    2564 scope.go:117] "RemoveContainer" containerID="bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4"
Jul 06 23:50:14.961051 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.961032    2564 scope.go:117] "RemoveContainer" containerID="bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4"
Jul 06 23:50:14.961327 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:14.961309    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4\": container with ID starting with bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4 not found: ID does not exist" containerID="bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4"
Jul 06 23:50:14.961401 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.961336    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4"} err="failed to get container status \"bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4\": rpc error: code = NotFound desc = could not find container \"bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4\": container with ID starting with bd0366a3683fcae6297427a1ba4537c7cc3cf7b45fc85e590f6f49dc42d848e4 not found: ID does not exist"
Jul 06 23:50:14.974215 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.974187    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689645f56f-bj2lp"]
Jul 06 23:50:14.978145 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:14.978121    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-689645f56f-bj2lp"]
Jul 06 23:50:15.389776 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:15.389747    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23e63bf0-0f87-49d8-bfd8-34bbad9dc866" path="/var/lib/kubelet/pods/23e63bf0-0f87-49d8-bfd8-34bbad9dc866/volumes"
Jul 06 23:50:28.209439 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:28.209397    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:28.229181 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:28.229153    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:29.019927 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:29.019900    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:41.939247 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:41.939211    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:50:41.940146 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:41.940083    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="alertmanager" containerID="cri-o://eb892f79e5c7ba11217929d018d981940e6c002b9cba6404459e45587af015ff" gracePeriod=120
Jul 06 23:50:41.940330 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:41.940137    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-metric" containerID="cri-o://3bbfcc67bd371f57a4f9661f7edc4ff2f13891024ed6179a6b6e76f41a25fbb7" gracePeriod=120
Jul 06 23:50:41.940330 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:41.940173    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-web" containerID="cri-o://e20007f48edded56ab85ffaed092544b2f7a944557abba0803e33c32d64f2958" gracePeriod=120
Jul 06 23:50:41.940330 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:41.940203    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy" containerID="cri-o://25c7a31fae8f0ea82cbf4c76d2bca606812d51c688c8cc75a01884eef8d1155a" gracePeriod=120
Jul 06 23:50:41.940330 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:41.940205    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="prom-label-proxy" containerID="cri-o://5037369be723a2ae133b4b0f68c2feede9c1c02974874ed475de045270c5cc59" gracePeriod=120
Jul 06 23:50:41.940609 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:41.940303    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="config-reloader" containerID="cri-o://83bbd3d23941d268742d75844605c499b5cb05d1f7c2818426020fd67a02df34" gracePeriod=120
Jul 06 23:50:43.057812 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057779    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerID="5037369be723a2ae133b4b0f68c2feede9c1c02974874ed475de045270c5cc59" exitCode=0
Jul 06 23:50:43.057812 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057806    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerID="3bbfcc67bd371f57a4f9661f7edc4ff2f13891024ed6179a6b6e76f41a25fbb7" exitCode=0
Jul 06 23:50:43.057812 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057814    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerID="25c7a31fae8f0ea82cbf4c76d2bca606812d51c688c8cc75a01884eef8d1155a" exitCode=0
Jul 06 23:50:43.057812 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057819    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerID="e20007f48edded56ab85ffaed092544b2f7a944557abba0803e33c32d64f2958" exitCode=0
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057824    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerID="83bbd3d23941d268742d75844605c499b5cb05d1f7c2818426020fd67a02df34" exitCode=0
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057829    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerID="eb892f79e5c7ba11217929d018d981940e6c002b9cba6404459e45587af015ff" exitCode=0
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"5037369be723a2ae133b4b0f68c2feede9c1c02974874ed475de045270c5cc59"}
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057921    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"3bbfcc67bd371f57a4f9661f7edc4ff2f13891024ed6179a6b6e76f41a25fbb7"}
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"25c7a31fae8f0ea82cbf4c76d2bca606812d51c688c8cc75a01884eef8d1155a"}
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.057977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"e20007f48edded56ab85ffaed092544b2f7a944557abba0803e33c32d64f2958"}
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.058002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"83bbd3d23941d268742d75844605c499b5cb05d1f7c2818426020fd67a02df34"}
Jul 06 23:50:43.058378 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.058025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"eb892f79e5c7ba11217929d018d981940e6c002b9cba6404459e45587af015ff"}
Jul 06 23:50:43.194615 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.194591    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:50:43.269300 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-main-db\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269300 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269254    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269300 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269282    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269312    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcxc9\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-kube-api-access-rcxc9\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269337    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-cluster-tls-config\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269360    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269385    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-out\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269453    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-metrics-client-ca\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269482    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269509    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-volume\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269546    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-web-config\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.269595 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269582    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-tls-assets\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.270190 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269613    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-web\") pod \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\" (UID: \"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311\") "
Jul 06 23:50:43.270190 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269593    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:50:43.270190 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269755    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:43.270190 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269867    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-main-db\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.270190 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.269887    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.270826 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.270795    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:43.273926 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.273897    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:43.274589 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.274552    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:43.274692 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.274626    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-out" (OuterVolumeSpecName: "config-out") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:50:43.274752 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.274693    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:43.274848 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.274814    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:43.275065 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.275036    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:50:43.275163 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.275075    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-volume" (OuterVolumeSpecName: "config-volume") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:43.275163 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.275115    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-kube-api-access-rcxc9" (OuterVolumeSpecName: "kube-api-access-rcxc9") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "kube-api-access-rcxc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:50:43.277619 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.277504    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:43.284711 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.284686    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-web-config" (OuterVolumeSpecName: "web-config") pod "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" (UID: "7f56bb2d-96c8-4ccc-a2a3-8df2d243b311"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:43.371233 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371177    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-tls-assets\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371233 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371223    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371233 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371234    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371233 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371245    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371233 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371255    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcxc9\" (UniqueName: \"kubernetes.io/projected/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-kube-api-access-rcxc9\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371263    2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-cluster-tls-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371271    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-out\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371279    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-metrics-client-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371286    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-secret-alertmanager-main-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371297    2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-config-volume\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:43.371579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:43.371307    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311-web-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:44.064490 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.064456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7f56bb2d-96c8-4ccc-a2a3-8df2d243b311","Type":"ContainerDied","Data":"03c4d3ca1f1dcd43df7af6e045d380c8336f2c99f659427ac41b2df1a1b0f66b"}
Jul 06 23:50:44.064901 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.064523    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:50:44.064901 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.064543    2564 scope.go:117] "RemoveContainer" containerID="5037369be723a2ae133b4b0f68c2feede9c1c02974874ed475de045270c5cc59"
Jul 06 23:50:44.072095 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.072076    2564 scope.go:117] "RemoveContainer" containerID="3bbfcc67bd371f57a4f9661f7edc4ff2f13891024ed6179a6b6e76f41a25fbb7"
Jul 06 23:50:44.079599 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.079582    2564 scope.go:117] "RemoveContainer" containerID="25c7a31fae8f0ea82cbf4c76d2bca606812d51c688c8cc75a01884eef8d1155a"
Jul 06 23:50:44.087085 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.087027    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:50:44.087555 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.087533    2564 scope.go:117] "RemoveContainer" containerID="e20007f48edded56ab85ffaed092544b2f7a944557abba0803e33c32d64f2958"
Jul 06 23:50:44.094736 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.094712    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:50:44.095642 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.095619    2564 scope.go:117] "RemoveContainer" containerID="83bbd3d23941d268742d75844605c499b5cb05d1f7c2818426020fd67a02df34"
Jul 06 23:50:44.104576 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.104557    2564 scope.go:117] "RemoveContainer" containerID="eb892f79e5c7ba11217929d018d981940e6c002b9cba6404459e45587af015ff"
Jul 06 23:50:44.112404 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:44.112384    2564 scope.go:117] "RemoveContainer" containerID="f03a33893ed9da384df96ce0e6aa6000bf80e191823a025f6cac9baa22999444"
Jul 06 23:50:45.392700 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:45.390301    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" path="/var/lib/kubelet/pods/7f56bb2d-96c8-4ccc-a2a3-8df2d243b311/volumes"
Jul 06 23:50:46.172968 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.172933    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:50:46.173444 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.173388    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="prometheus" containerID="cri-o://9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81" gracePeriod=600
Jul 06 23:50:46.173574 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.173452    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-web" containerID="cri-o://a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab" gracePeriod=600
Jul 06 23:50:46.173574 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.173489    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979" gracePeriod=600
Jul 06 23:50:46.173574 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.173514    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="config-reloader" containerID="cri-o://7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a" gracePeriod=600
Jul 06 23:50:46.173744 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.173400    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="thanos-sidecar" containerID="cri-o://9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734" gracePeriod=600
Jul 06 23:50:46.173744 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.173400    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy" containerID="cri-o://dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6" gracePeriod=600
Jul 06 23:50:46.426169 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.425817    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:46.497363 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497323    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-kubelet-serving-ca-bundle\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497363 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497362    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-thanos-prometheus-http-client-file\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497387    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-web-config\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-0\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497468    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-tls-assets\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497499    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-db\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497534    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-metrics-client-certs\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497562    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-1\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497589    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.497626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497616    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-kube-rbac-proxy\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497659    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-serving-certs-ca-bundle\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497692    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config-out\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497716    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497724    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9s8b\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-kube-api-access-v9s8b\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497770    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-trusted-ca-bundle\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497805    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-grpc-tls\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497842    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497870    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-2\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497907    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497932    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-metrics-client-ca\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.497957    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-tls\") pod \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\" (UID: \"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77\") "
Jul 06 23:50:46.498028 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.498020    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:46.498628 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.498299    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.498628 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.498319    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.501341 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.500021    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:46.501341 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.500306    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:46.501341 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.500441    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:46.501341 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.500819    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:46.501341 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.501329    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:50:46.503105 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.502866    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:50:46.504032 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.503988    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.504136 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.504073    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.504795 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.504756    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.505294 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.505259    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config" (OuterVolumeSpecName: "config") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.505392 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.505363    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.506009 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.505983    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-kube-api-access-v9s8b" (OuterVolumeSpecName: "kube-api-access-v9s8b") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "kube-api-access-v9s8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:50:46.506118 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.506027    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.506331 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.506285    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.506608 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.506579    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:50:46.506938 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.506907    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config-out" (OuterVolumeSpecName: "config-out") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:50:46.507493 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.507467    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.516704 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.516674    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-web-config" (OuterVolumeSpecName: "web-config") pod "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" (UID: "9b6b12b1-e3ce-4d72-89ef-fa82563f7b77"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:50:46.599196 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599158    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config-out\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599196 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599192    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9s8b\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-kube-api-access-v9s8b\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599196 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599204    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599215    2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-grpc-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599224    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599233    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599242    2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599249    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-metrics-client-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599258    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599267    2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599275    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-web-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599283    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599291    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-tls-assets\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599299    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-prometheus-k8s-db\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599307    2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-metrics-client-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599315    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599323    2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-secret-kube-rbac-proxy\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:46.599455 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:46.599331    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:50:47.078168 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078135    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979" exitCode=0
Jul 06 23:50:47.078168 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078161    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6" exitCode=0
Jul 06 23:50:47.078168 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078172    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab" exitCode=0
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078180    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734" exitCode=0
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078187    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a" exitCode=0
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078194    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81" exitCode=0
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078204    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"}
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078293    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"}
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078321    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"}
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078346    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"}
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078365    2564 scope.go:117] "RemoveContainer" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078369    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"}
Jul 06 23:50:47.078473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"}
Jul 06 23:50:47.078942 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.078488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b6b12b1-e3ce-4d72-89ef-fa82563f7b77","Type":"ContainerDied","Data":"977972ba5cd70217d9568bad9fac529a6f4c42e23a31b95585e00e82b25817da"}
Jul 06 23:50:47.086739 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.086718    2564 scope.go:117] "RemoveContainer" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.093698 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.093680    2564 scope.go:117] "RemoveContainer" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.100323 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.100304    2564 scope.go:117] "RemoveContainer" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.101296 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.101271    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:50:47.106116 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.106094    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:50:47.107754 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.107738    2564 scope.go:117] "RemoveContainer" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.114762 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.114744    2564 scope.go:117] "RemoveContainer" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.121948 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.121925    2564 scope.go:117] "RemoveContainer" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.129116 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.129093    2564 scope.go:117] "RemoveContainer" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.129382 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:47.129366    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": container with ID starting with f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979 not found: ID does not exist" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.129452 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.129394    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"} err="failed to get container status \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": rpc error: code = NotFound desc = could not find container \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": container with ID starting with f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979 not found: ID does not exist"
Jul 06 23:50:47.129452 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.129412    2564 scope.go:117] "RemoveContainer" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.129648 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:47.129630    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": container with ID starting with dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6 not found: ID does not exist" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.129707 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.129657    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"} err="failed to get container status \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": rpc error: code = NotFound desc = could not find container \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": container with ID starting with dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6 not found: ID does not exist"
Jul 06 23:50:47.129707 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.129686    2564 scope.go:117] "RemoveContainer" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.129914 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:47.129897    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": container with ID starting with a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab not found: ID does not exist" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.129966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.129920    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"} err="failed to get container status \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": rpc error: code = NotFound desc = could not find container \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": container with ID starting with a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab not found: ID does not exist"
Jul 06 23:50:47.129966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.129938    2564 scope.go:117] "RemoveContainer" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.130238 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:47.130218    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": container with ID starting with 9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734 not found: ID does not exist" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.130277 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130243    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"} err="failed to get container status \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": rpc error: code = NotFound desc = could not find container \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": container with ID starting with 9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734 not found: ID does not exist"
Jul 06 23:50:47.130277 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130259    2564 scope.go:117] "RemoveContainer" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.130494 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:47.130476    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": container with ID starting with 7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a not found: ID does not exist" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.130544 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130498    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"} err="failed to get container status \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": rpc error: code = NotFound desc = could not find container \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": container with ID starting with 7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a not found: ID does not exist"
Jul 06 23:50:47.130544 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130510    2564 scope.go:117] "RemoveContainer" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.130747 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:47.130730    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": container with ID starting with 9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81 not found: ID does not exist" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.130779 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130751    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"} err="failed to get container status \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": rpc error: code = NotFound desc = could not find container \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": container with ID starting with 9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81 not found: ID does not exist"
Jul 06 23:50:47.130779 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130763    2564 scope.go:117] "RemoveContainer" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.130963 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:50:47.130946    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": container with ID starting with b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05 not found: ID does not exist" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.131002 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130966    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"} err="failed to get container status \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": rpc error: code = NotFound desc = could not find container \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": container with ID starting with b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05 not found: ID does not exist"
Jul 06 23:50:47.131002 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.130977    2564 scope.go:117] "RemoveContainer" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.131192 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131175    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"} err="failed to get container status \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": rpc error: code = NotFound desc = could not find container \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": container with ID starting with f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979 not found: ID does not exist"
Jul 06 23:50:47.131245 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131194    2564 scope.go:117] "RemoveContainer" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.131459 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131415    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"} err="failed to get container status \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": rpc error: code = NotFound desc = could not find container \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": container with ID starting with dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6 not found: ID does not exist"
Jul 06 23:50:47.131510 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131459    2564 scope.go:117] "RemoveContainer" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.131694 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131668    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"} err="failed to get container status \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": rpc error: code = NotFound desc = could not find container \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": container with ID starting with a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab not found: ID does not exist"
Jul 06 23:50:47.131694 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131688    2564 scope.go:117] "RemoveContainer" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.131921 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131907    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"} err="failed to get container status \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": rpc error: code = NotFound desc = could not find container \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": container with ID starting with 9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734 not found: ID does not exist"
Jul 06 23:50:47.131957 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.131923    2564 scope.go:117] "RemoveContainer" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.132128 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132114    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"} err="failed to get container status \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": rpc error: code = NotFound desc = could not find container \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": container with ID starting with 7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a not found: ID does not exist"
Jul 06 23:50:47.132167 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132128    2564 scope.go:117] "RemoveContainer" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.132347 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132328    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"} err="failed to get container status \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": rpc error: code = NotFound desc = could not find container \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": container with ID starting with 9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81 not found: ID does not exist"
Jul 06 23:50:47.132384 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132348    2564 scope.go:117] "RemoveContainer" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.132564 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132541    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"} err="failed to get container status \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": rpc error: code = NotFound desc = could not find container \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": container with ID starting with b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05 not found: ID does not exist"
Jul 06 23:50:47.132650 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132566    2564 scope.go:117] "RemoveContainer" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.132775 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132757    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"} err="failed to get container status \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": rpc error: code = NotFound desc = could not find container \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": container with ID starting with f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979 not found: ID does not exist"
Jul 06 23:50:47.132812 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132775    2564 scope.go:117] "RemoveContainer" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.132992 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132974    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"} err="failed to get container status \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": rpc error: code = NotFound desc = could not find container \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": container with ID starting with dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6 not found: ID does not exist"
Jul 06 23:50:47.133029 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.132992    2564 scope.go:117] "RemoveContainer" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.133214 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133196    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"} err="failed to get container status \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": rpc error: code = NotFound desc = could not find container \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": container with ID starting with a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab not found: ID does not exist"
Jul 06 23:50:47.133257 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133215    2564 scope.go:117] "RemoveContainer" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.133417 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133399    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"} err="failed to get container status \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": rpc error: code = NotFound desc = could not find container \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": container with ID starting with 9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734 not found: ID does not exist"
Jul 06 23:50:47.133417 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133417    2564 scope.go:117] "RemoveContainer" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.133677 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133660    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"} err="failed to get container status \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": rpc error: code = NotFound desc = could not find container \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": container with ID starting with 7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a not found: ID does not exist"
Jul 06 23:50:47.133717 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133677    2564 scope.go:117] "RemoveContainer" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.133894 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133876    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"} err="failed to get container status \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": rpc error: code = NotFound desc = could not find container \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": container with ID starting with 9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81 not found: ID does not exist"
Jul 06 23:50:47.133940 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.133895    2564 scope.go:117] "RemoveContainer" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.134085 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.134067    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"} err="failed to get container status \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": rpc error: code = NotFound desc = could not find container \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": container with ID starting with b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05 not found: ID does not exist"
Jul 06 23:50:47.134136 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.134085    2564 scope.go:117] "RemoveContainer" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.134291 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.134274    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"} err="failed to get container status \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": rpc error: code = NotFound desc = could not find container \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": container with ID starting with f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979 not found: ID does not exist"
Jul 06 23:50:47.134342 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.134291    2564 scope.go:117] "RemoveContainer" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.134648 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.134618    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"} err="failed to get container status \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": rpc error: code = NotFound desc = could not find container \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": container with ID starting with dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6 not found: ID does not exist"
Jul 06 23:50:47.134648 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.134644    2564 scope.go:117] "RemoveContainer" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.135183 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.135080    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"} err="failed to get container status \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": rpc error: code = NotFound desc = could not find container \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": container with ID starting with a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab not found: ID does not exist"
Jul 06 23:50:47.135183 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.135107    2564 scope.go:117] "RemoveContainer" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.135777 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.135749    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"} err="failed to get container status \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": rpc error: code = NotFound desc = could not find container \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": container with ID starting with 9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734 not found: ID does not exist"
Jul 06 23:50:47.135887 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.135778    2564 scope.go:117] "RemoveContainer" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.136124 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136094    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"} err="failed to get container status \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": rpc error: code = NotFound desc = could not find container \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": container with ID starting with 7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a not found: ID does not exist"
Jul 06 23:50:47.136124 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136124    2564 scope.go:117] "RemoveContainer" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.136388 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136365    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"} err="failed to get container status \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": rpc error: code = NotFound desc = could not find container \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": container with ID starting with 9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81 not found: ID does not exist"
Jul 06 23:50:47.136473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136392    2564 scope.go:117] "RemoveContainer" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.136600 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136583    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:50:47.136686 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136665    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"} err="failed to get container status \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": rpc error: code = NotFound desc = could not find container \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": container with ID starting with b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05 not found: ID does not exist"
Jul 06 23:50:47.136751 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136688    2564 scope.go:117] "RemoveContainer" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.136909 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136890    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="alertmanager"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136912    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="alertmanager"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136926    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23e63bf0-0f87-49d8-bfd8-34bbad9dc866" containerName="console"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136935    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="23e63bf0-0f87-49d8-bfd8-34bbad9dc866" containerName="console"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136947    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52106a2b-62a9-4ada-b374-d0f70bbd7f7d" containerName="console"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136955    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="52106a2b-62a9-4ada-b374-d0f70bbd7f7d" containerName="console"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136964    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="prometheus"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136970    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="prometheus"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136979    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136986    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136994    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="config-reloader"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137003    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="config-reloader"
Jul 06 23:50:47.137017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.136928    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"} err="failed to get container status \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": rpc error: code = NotFound desc = could not find container \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": container with ID starting with f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979 not found: ID does not exist"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137026    2564 scope.go:117] "RemoveContainer" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137013    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-thanos"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137091    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-thanos"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137107    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-web"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137116    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-web"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137129    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-metric"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137139    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-metric"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137145    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="init-config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137151    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="init-config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137158    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="thanos-sidecar"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137162    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="thanos-sidecar"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137168    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-web"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137175    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-web"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137187    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="prom-label-proxy"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137195    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="prom-label-proxy"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137204    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137212    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137225    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137233    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137244    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="init-config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137253    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="init-config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137315    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137328    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-metric"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137336    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="prometheus"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137341    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-web"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137347    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy-web"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137351    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137357    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="52106a2b-62a9-4ada-b374-d0f70bbd7f7d" containerName="console"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137363    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="23e63bf0-0f87-49d8-bfd8-34bbad9dc866" containerName="console"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137367    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="alertmanager"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137373    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="config-reloader"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137379    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="thanos-sidecar"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137384    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" containerName="kube-rbac-proxy-thanos"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137389    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="kube-rbac-proxy"
Jul 06 23:50:47.137575 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.137395    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56bb2d-96c8-4ccc-a2a3-8df2d243b311" containerName="prom-label-proxy"
Jul 06 23:50:47.139254 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.139229    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"} err="failed to get container status \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": rpc error: code = NotFound desc = could not find container \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": container with ID starting with dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6 not found: ID does not exist"
Jul 06 23:50:47.139254 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.139254    2564 scope.go:117] "RemoveContainer" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.139631 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.139607    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"} err="failed to get container status \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": rpc error: code = NotFound desc = could not find container \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": container with ID starting with a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab not found: ID does not exist"
Jul 06 23:50:47.139631 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.139630    2564 scope.go:117] "RemoveContainer" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.139882 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.139864    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"} err="failed to get container status \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": rpc error: code = NotFound desc = could not find container \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": container with ID starting with 9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734 not found: ID does not exist"
Jul 06 23:50:47.139937 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.139885    2564 scope.go:117] "RemoveContainer" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.140110 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140094    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"} err="failed to get container status \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": rpc error: code = NotFound desc = could not find container \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": container with ID starting with 7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a not found: ID does not exist"
Jul 06 23:50:47.140164 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140120    2564 scope.go:117] "RemoveContainer" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.140309 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140296    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"} err="failed to get container status \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": rpc error: code = NotFound desc = could not find container \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": container with ID starting with 9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81 not found: ID does not exist"
Jul 06 23:50:47.140343 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140310    2564 scope.go:117] "RemoveContainer" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.140504 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140485    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"} err="failed to get container status \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": rpc error: code = NotFound desc = could not find container \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": container with ID starting with b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05 not found: ID does not exist"
Jul 06 23:50:47.140551 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140504    2564 scope.go:117] "RemoveContainer" containerID="f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"
Jul 06 23:50:47.140705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140691    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979"} err="failed to get container status \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": rpc error: code = NotFound desc = could not find container \"f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979\": container with ID starting with f10b70bcc9b7998a1cc8c4256b75ddf6b9a39b00c0288bb61a275f27f9393979 not found: ID does not exist"
Jul 06 23:50:47.140747 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140706    2564 scope.go:117] "RemoveContainer" containerID="dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"
Jul 06 23:50:47.140906 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140887    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6"} err="failed to get container status \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": rpc error: code = NotFound desc = could not find container \"dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6\": container with ID starting with dafff030cf1fcf0882dafc16e6073513fff39099831819fda312e0d81aa6a7f6 not found: ID does not exist"
Jul 06 23:50:47.140950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.140909    2564 scope.go:117] "RemoveContainer" containerID="a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"
Jul 06 23:50:47.141124 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141109    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab"} err="failed to get container status \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": rpc error: code = NotFound desc = could not find container \"a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab\": container with ID starting with a642a2f54e135d54004d1dde6789debd728c5d4d87afec5b03030d0dbee644ab not found: ID does not exist"
Jul 06 23:50:47.141169 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141124    2564 scope.go:117] "RemoveContainer" containerID="9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"
Jul 06 23:50:47.141310 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141295    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734"} err="failed to get container status \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": rpc error: code = NotFound desc = could not find container \"9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734\": container with ID starting with 9ffe5d999b127af41960038a4c4a39334b23553c2da9d083c60193305ed2a734 not found: ID does not exist"
Jul 06 23:50:47.141349 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141309    2564 scope.go:117] "RemoveContainer" containerID="7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"
Jul 06 23:50:47.141537 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141516    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a"} err="failed to get container status \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": rpc error: code = NotFound desc = could not find container \"7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a\": container with ID starting with 7d7094be6521fb2a677c25635acf88eb9ae224961eb3b748103b3ffcffdc416a not found: ID does not exist"
Jul 06 23:50:47.141537 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141536    2564 scope.go:117] "RemoveContainer" containerID="9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"
Jul 06 23:50:47.141730 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141714    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81"} err="failed to get container status \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": rpc error: code = NotFound desc = could not find container \"9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81\": container with ID starting with 9e05f68140547ab97eab982c3c28c4b1ea297b538dd461c759826b648d2b4a81 not found: ID does not exist"
Jul 06 23:50:47.141771 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141731    2564 scope.go:117] "RemoveContainer" containerID="b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"
Jul 06 23:50:47.141921 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.141905    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05"} err="failed to get container status \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": rpc error: code = NotFound desc = could not find container \"b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05\": container with ID starting with b5860e710828a184425c84619f274ec5a5c99074643183b5bdca92ec287ead05 not found: ID does not exist"
Jul 06 23:50:47.142758 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.142744    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.145652 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.145623    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 23:50:47.145766 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.145623    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4mcl4t0upgubi\""
Jul 06 23:50:47.145766 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.145705    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 23:50:47.145766 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.145623    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 23:50:47.145766 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.145715    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 23:50:47.145944 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.145765    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-g4xph\""
Jul 06 23:50:47.146484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146224    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 23:50:47.146484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146251    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 23:50:47.146484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146259    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 23:50:47.146484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146276    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 23:50:47.146484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146252    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 23:50:47.146484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146225    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 23:50:47.146829 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146814    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 23:50:47.146884 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.146855    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 23:50:47.149546 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.149528    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 23:50:47.151250 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.151232    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 23:50:47.156900 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.156877    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:50:47.202949 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.202909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.202949 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.202951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203122 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.202967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203122 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203122 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203122 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203122 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203261 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203261 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203261 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203172    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-config\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203261 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203261 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203207    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203261 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203270    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md5br\" (UniqueName: \"kubernetes.io/projected/0908b5df-02f9-477d-a376-52bca100223b-kube-api-access-md5br\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0908b5df-02f9-477d-a376-52bca100223b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0908b5df-02f9-477d-a376-52bca100223b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.203446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.203385    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.304651 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.304651 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304797    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md5br\" (UniqueName: \"kubernetes.io/projected/0908b5df-02f9-477d-a376-52bca100223b-kube-api-access-md5br\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0908b5df-02f9-477d-a376-52bca100223b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0908b5df-02f9-477d-a376-52bca100223b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.304980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305065    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305187    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-config\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.305948 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.305769    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.306065 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.306042    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.306129 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.306059    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.306473 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.306453    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.308510 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.308113    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0908b5df-02f9-477d-a376-52bca100223b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.308510 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.308137    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.308510 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.308137    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.308730 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.308564    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.308788 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.308684    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.309404 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.309375    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.309713 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.309694    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.309945 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.309921    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0908b5df-02f9-477d-a376-52bca100223b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.310062 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.309951    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.310214 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.310169    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.310614 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.310584    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.310849 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.310830    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.310910 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.310868    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.311461 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.311400    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0908b5df-02f9-477d-a376-52bca100223b-config\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.311641 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.311590    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0908b5df-02f9-477d-a376-52bca100223b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.313005 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.312985    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md5br\" (UniqueName: \"kubernetes.io/projected/0908b5df-02f9-477d-a376-52bca100223b-kube-api-access-md5br\") pod \"prometheus-k8s-0\" (UID: \"0908b5df-02f9-477d-a376-52bca100223b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.390456 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.390343    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6b12b1-e3ce-4d72-89ef-fa82563f7b77" path="/var/lib/kubelet/pods/9b6b12b1-e3ce-4d72-89ef-fa82563f7b77/volumes"
Jul 06 23:50:47.453166 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.453128    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:47.599727 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:47.599700    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:50:47.602220 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:50:47.602194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0908b5df_02f9_477d_a376_52bca100223b.slice/crio-75dcc07d5c4f77af5b4feb0f4d15b596d5459c403d622c4c3b389344631538c7 WatchSource:0}: Error finding container 75dcc07d5c4f77af5b4feb0f4d15b596d5459c403d622c4c3b389344631538c7: Status 404 returned error can't find the container with id 75dcc07d5c4f77af5b4feb0f4d15b596d5459c403d622c4c3b389344631538c7
Jul 06 23:50:48.084415 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:48.084377    2564 generic.go:356] "Generic (PLEG): container finished" podID="0908b5df-02f9-477d-a376-52bca100223b" containerID="35ce7a6e662b86336b2a0858ac559e90938a4c2d432c5f17d2771ef5253dd331" exitCode=0
Jul 06 23:50:48.084573 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:48.084471    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerDied","Data":"35ce7a6e662b86336b2a0858ac559e90938a4c2d432c5f17d2771ef5253dd331"}
Jul 06 23:50:48.084573 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:48.084558    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerStarted","Data":"75dcc07d5c4f77af5b4feb0f4d15b596d5459c403d622c4c3b389344631538c7"}
Jul 06 23:50:49.092578 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:49.092543    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerStarted","Data":"16f9bf3ef6bab5fc75c8453b4d3ed624a7cf26caf059fd486bca5ac034114742"}
Jul 06 23:50:49.092960 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:49.092614    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerStarted","Data":"311878450cf5a055c22540afc992b6af0a69920fc66c7f7fb72a0b7f074f99a7"}
Jul 06 23:50:49.092960 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:49.092641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerStarted","Data":"77354e744a6f6a815a981aeda935560425b23e366e825dc260a1aa7cc1c29a61"}
Jul 06 23:50:49.092960 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:49.092670    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerStarted","Data":"ce352d7f970814d3ba0c38bb92e519c4acf77434ad3615c773969d263d03e2a8"}
Jul 06 23:50:49.092960 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:49.092702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerStarted","Data":"7c789c51b9670cff132fc6c393ec61174df0292199cc4615ca95f0adea649b4d"}
Jul 06 23:50:49.092960 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:49.092725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0908b5df-02f9-477d-a376-52bca100223b","Type":"ContainerStarted","Data":"7164fa7a090536692ac3143b26b0e2cff3f8248f16b28b25dd3218dd7d643b3a"}
Jul 06 23:50:49.122966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:49.122907    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.103592969 podStartE2EDuration="2.122891352s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.298383ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:49.120412281 +0000 UTC m=+150.185744260" watchObservedRunningTime="2026-07-06 23:50:49.122891352 +0000 UTC m=+150.188223329"
Jul 06 23:50:52.454116 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:52.454078    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:50:52.916875 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:50:52.916832    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76b4b8687b-5cqqt"]
Jul 06 23:51:17.938016 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:17.937972    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76b4b8687b-5cqqt" podUID="07f0a593-ecd0-4c29-8269-c478971ca94b" containerName="console" containerID="cri-o://7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf" gracePeriod=15
Jul 06 23:51:18.174130 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.174106    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:51:18.195708 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.195630    2564 generic.go:356] "Generic (PLEG): container finished" podID="07f0a593-ecd0-4c29-8269-c478971ca94b" containerID="7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf" exitCode=0
Jul 06 23:51:18.195944 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.195892    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b4b8687b-5cqqt"
Jul 06 23:51:18.195944 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.195812    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b4b8687b-5cqqt" event={"ID":"07f0a593-ecd0-4c29-8269-c478971ca94b","Type":"ContainerDied","Data":"7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf"}
Jul 06 23:51:18.196201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.195967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b4b8687b-5cqqt" event={"ID":"07f0a593-ecd0-4c29-8269-c478971ca94b","Type":"ContainerDied","Data":"ec830534d6a48ddff596de962860354bbbf26ab0eac3dfa965dee1c84c7dfe79"}
Jul 06 23:51:18.196201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.196001    2564 scope.go:117] "RemoveContainer" containerID="7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf"
Jul 06 23:51:18.205288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.205272    2564 scope.go:117] "RemoveContainer" containerID="7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf"
Jul 06 23:51:18.205600 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:51:18.205580    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf\": container with ID starting with 7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf not found: ID does not exist" containerID="7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf"
Jul 06 23:51:18.205650 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.205608    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf"} err="failed to get container status \"7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf\": rpc error: code = NotFound desc = could not find container \"7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf\": container with ID starting with 7561450083dc3d9ed8bff4b6812508a6cd135886fe8774ea5202d249f3668ccf not found: ID does not exist"
Jul 06 23:51:18.252518 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252492    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-oauth-config\") pod \"07f0a593-ecd0-4c29-8269-c478971ca94b\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") "
Jul 06 23:51:18.252518 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252521    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-oauth-serving-cert\") pod \"07f0a593-ecd0-4c29-8269-c478971ca94b\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") "
Jul 06 23:51:18.252703 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252546    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-service-ca\") pod \"07f0a593-ecd0-4c29-8269-c478971ca94b\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") "
Jul 06 23:51:18.252703 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252584    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzqgz\" (UniqueName: \"kubernetes.io/projected/07f0a593-ecd0-4c29-8269-c478971ca94b-kube-api-access-bzqgz\") pod \"07f0a593-ecd0-4c29-8269-c478971ca94b\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") "
Jul 06 23:51:18.252703 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252603    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-console-config\") pod \"07f0a593-ecd0-4c29-8269-c478971ca94b\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") "
Jul 06 23:51:18.252703 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252627    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-serving-cert\") pod \"07f0a593-ecd0-4c29-8269-c478971ca94b\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") "
Jul 06 23:51:18.252703 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252672    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-trusted-ca-bundle\") pod \"07f0a593-ecd0-4c29-8269-c478971ca94b\" (UID: \"07f0a593-ecd0-4c29-8269-c478971ca94b\") "
Jul 06 23:51:18.253049 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.252950    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07f0a593-ecd0-4c29-8269-c478971ca94b" (UID: "07f0a593-ecd0-4c29-8269-c478971ca94b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:51:18.253150 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.253128    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "07f0a593-ecd0-4c29-8269-c478971ca94b" (UID: "07f0a593-ecd0-4c29-8269-c478971ca94b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:51:18.253452 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.253062    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-console-config" (OuterVolumeSpecName: "console-config") pod "07f0a593-ecd0-4c29-8269-c478971ca94b" (UID: "07f0a593-ecd0-4c29-8269-c478971ca94b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:51:18.253452 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.253075    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-service-ca" (OuterVolumeSpecName: "service-ca") pod "07f0a593-ecd0-4c29-8269-c478971ca94b" (UID: "07f0a593-ecd0-4c29-8269-c478971ca94b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:51:18.254800 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.254774    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f0a593-ecd0-4c29-8269-c478971ca94b-kube-api-access-bzqgz" (OuterVolumeSpecName: "kube-api-access-bzqgz") pod "07f0a593-ecd0-4c29-8269-c478971ca94b" (UID: "07f0a593-ecd0-4c29-8269-c478971ca94b"). InnerVolumeSpecName "kube-api-access-bzqgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:51:18.254880 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.254793    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07f0a593-ecd0-4c29-8269-c478971ca94b" (UID: "07f0a593-ecd0-4c29-8269-c478971ca94b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:51:18.254880 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.254811    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07f0a593-ecd0-4c29-8269-c478971ca94b" (UID: "07f0a593-ecd0-4c29-8269-c478971ca94b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:51:18.353379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.353340    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-oauth-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:51:18.353379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.353366    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-oauth-serving-cert\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:51:18.353379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.353375    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:51:18.353379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.353383    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzqgz\" (UniqueName: \"kubernetes.io/projected/07f0a593-ecd0-4c29-8269-c478971ca94b-kube-api-access-bzqgz\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:51:18.353379 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.353392    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-console-config\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:51:18.353776 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.353401    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f0a593-ecd0-4c29-8269-c478971ca94b-console-serving-cert\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:51:18.353776 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.353409    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f0a593-ecd0-4c29-8269-c478971ca94b-trusted-ca-bundle\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:51:18.516689 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.516658    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76b4b8687b-5cqqt"]
Jul 06 23:51:18.523850 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:18.523828    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76b4b8687b-5cqqt"]
Jul 06 23:51:19.389124 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:19.389092    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f0a593-ecd0-4c29-8269-c478971ca94b" path="/var/lib/kubelet/pods/07f0a593-ecd0-4c29-8269-c478971ca94b/volumes"
Jul 06 23:51:47.453850 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:47.453809    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:51:47.470866 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:47.470840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:51:48.317706 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:48.317680    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:51:54.446372 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.446338    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qf867"]
Jul 06 23:51:54.446829 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.446602    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f0a593-ecd0-4c29-8269-c478971ca94b" containerName="console"
Jul 06 23:51:54.446829 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.446615    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f0a593-ecd0-4c29-8269-c478971ca94b" containerName="console"
Jul 06 23:51:54.446829 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.446659    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f0a593-ecd0-4c29-8269-c478971ca94b" containerName="console"
Jul 06 23:51:54.449630 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.449614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.452486 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.452455    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 23:51:54.452693 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.452675    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-kkqmm\""
Jul 06 23:51:54.453633 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.453617    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 23:51:54.460106 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.460087    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qf867"]
Jul 06 23:51:54.535080 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.535043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq6n8\" (UniqueName: \"kubernetes.io/projected/6e05ccea-7aaa-49f1-b63e-b2160a5bb430-kube-api-access-cq6n8\") pod \"cert-manager-webhook-6486ffd997-qf867\" (UID: \"6e05ccea-7aaa-49f1-b63e-b2160a5bb430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.535080 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.535084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e05ccea-7aaa-49f1-b63e-b2160a5bb430-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qf867\" (UID: \"6e05ccea-7aaa-49f1-b63e-b2160a5bb430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.635411 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.635377    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq6n8\" (UniqueName: \"kubernetes.io/projected/6e05ccea-7aaa-49f1-b63e-b2160a5bb430-kube-api-access-cq6n8\") pod \"cert-manager-webhook-6486ffd997-qf867\" (UID: \"6e05ccea-7aaa-49f1-b63e-b2160a5bb430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.635411 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.635411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e05ccea-7aaa-49f1-b63e-b2160a5bb430-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qf867\" (UID: \"6e05ccea-7aaa-49f1-b63e-b2160a5bb430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.643472 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.643417    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e05ccea-7aaa-49f1-b63e-b2160a5bb430-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qf867\" (UID: \"6e05ccea-7aaa-49f1-b63e-b2160a5bb430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.643577 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.643489    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq6n8\" (UniqueName: \"kubernetes.io/projected/6e05ccea-7aaa-49f1-b63e-b2160a5bb430-kube-api-access-cq6n8\") pod \"cert-manager-webhook-6486ffd997-qf867\" (UID: \"6e05ccea-7aaa-49f1-b63e-b2160a5bb430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.773581 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.773543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:54.897520 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:54.897499    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qf867"]
Jul 06 23:51:54.900148 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:51:54.900117    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e05ccea_7aaa_49f1_b63e_b2160a5bb430.slice/crio-a3b82c4b2df288c41dd6cedcf59b844823dfd5bbae6c11ea1df7455ad3613931 WatchSource:0}: Error finding container a3b82c4b2df288c41dd6cedcf59b844823dfd5bbae6c11ea1df7455ad3613931: Status 404 returned error can't find the container with id a3b82c4b2df288c41dd6cedcf59b844823dfd5bbae6c11ea1df7455ad3613931
Jul 06 23:51:55.327689 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:55.327654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qf867" event={"ID":"6e05ccea-7aaa-49f1-b63e-b2160a5bb430","Type":"ContainerStarted","Data":"a3b82c4b2df288c41dd6cedcf59b844823dfd5bbae6c11ea1df7455ad3613931"}
Jul 06 23:51:58.341951 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:58.341867    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qf867" event={"ID":"6e05ccea-7aaa-49f1-b63e-b2160a5bb430","Type":"ContainerStarted","Data":"8df286bab907d21c73508f013b122816dee626b4af6bd27df045e4f3d2d4fe13"}
Jul 06 23:51:58.341951 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:58.341936    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:51:58.361164 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:51:58.361110    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qf867" podStartSLOduration=1.253978848 podStartE2EDuration="4.361094363s" totalImagesPullingTime="3.107115515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:51:58.360050215 +0000 UTC m=+219.425382193" watchObservedRunningTime="2026-07-06 23:51:58.361094363 +0000 UTC m=+219.426426339"
Jul 06 23:52:04.347657 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:04.347623    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qf867"
Jul 06 23:52:35.748076 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.748045    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"]
Jul 06 23:52:35.753127 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.753103    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.756050 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.756022    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 23:52:35.756172 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.756030    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 23:52:35.756172 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.756058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 23:52:35.756328 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.756314    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 23:52:35.756413 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.756398    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kmgzr\""
Jul 06 23:52:35.766824 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.766804    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"]
Jul 06 23:52:35.854369 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.854334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.854553 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.854381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-tmp\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.854553 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.854467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-manifests\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.854553 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.854493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdtpt\" (UniqueName: \"kubernetes.io/projected/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-kube-api-access-pdtpt\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.854553 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.854539    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.955704 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.955664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.955704 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.955711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.955950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.955744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-tmp\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.955950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.955775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-manifests\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.955950 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.955792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdtpt\" (UniqueName: \"kubernetes.io/projected/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-kube-api-access-pdtpt\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.956264 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.956231    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-manifests\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.958233 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.958213    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-tmp\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.958374 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.958358    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.958543 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.958525    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:35.964143 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:35.964119    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdtpt\" (UniqueName: \"kubernetes.io/projected/ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f-kube-api-access-pdtpt\") pod \"opendatahub-operator-controller-manager-6c4d578756-gj6kg\" (UID: \"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:36.063489 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:36.063380    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:36.213126 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:36.212175    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"]
Jul 06 23:52:36.469845 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:36.469803    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg" event={"ID":"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f","Type":"ContainerStarted","Data":"e292208bf9e5b5301ed077ce37b87d627e702e154358b8c63367e49663beeeac"}
Jul 06 23:52:39.483761 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:39.483731    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg" event={"ID":"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f","Type":"ContainerStarted","Data":"8d7032226ab6e01bb7fba36cd97afbdb64643796883f3789d8b1cdfa1e06f642"}
Jul 06 23:52:40.488550 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:40.488513    2564 generic.go:356] "Generic (PLEG): container finished" podID="ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f" containerID="8d7032226ab6e01bb7fba36cd97afbdb64643796883f3789d8b1cdfa1e06f642" exitCode=0
Jul 06 23:52:40.488965 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:40.488604    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg" event={"ID":"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f","Type":"ContainerDied","Data":"8d7032226ab6e01bb7fba36cd97afbdb64643796883f3789d8b1cdfa1e06f642"}
Jul 06 23:52:41.494092 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:41.494055    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg" event={"ID":"ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f","Type":"ContainerStarted","Data":"9dd3905520e507887dfbc48bb215372334ef9297bf0b2707e4082c62fe25659f"}
Jul 06 23:52:41.494721 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:41.494164    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:41.515500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:41.515447    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg" podStartSLOduration=2.280466373 podStartE2EDuration="6.515412713s" totalImagesPullingTime="3.198788671s" totalInitContainerRuntime="1.036157669s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:41.513880909 +0000 UTC m=+262.579212886" watchObservedRunningTime="2026-07-06 23:52:41.515412713 +0000 UTC m=+262.580744690"
Jul 06 23:52:52.504106 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:52.504076    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-gj6kg"
Jul 06 23:52:54.750351 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.750316    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"]
Jul 06 23:52:54.759605 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.759580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.763219 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.763198    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 23:52:54.763538 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.763523    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 23:52:54.764362 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.764349    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 23:52:54.764454 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.764417    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 23:52:54.764519 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.764457    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2ldws\""
Jul 06 23:52:54.782756 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.782728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"]
Jul 06 23:52:54.804465 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.804403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a778d1f2-e282-45bb-a9f9-26009fda6af0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.804694 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.804525    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbrk4\" (UniqueName: \"kubernetes.io/projected/a778d1f2-e282-45bb-a9f9-26009fda6af0-kube-api-access-jbrk4\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.804694 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.804563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a778d1f2-e282-45bb-a9f9-26009fda6af0-tmp\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.905188 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.905142    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a778d1f2-e282-45bb-a9f9-26009fda6af0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.905387 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.905199    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbrk4\" (UniqueName: \"kubernetes.io/projected/a778d1f2-e282-45bb-a9f9-26009fda6af0-kube-api-access-jbrk4\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.905387 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:52:54.905297    2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 06 23:52:54.905387 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.905358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a778d1f2-e282-45bb-a9f9-26009fda6af0-tmp\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.905387 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:52:54.905374    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a778d1f2-e282-45bb-a9f9-26009fda6af0-tls-certs podName:a778d1f2-e282-45bb-a9f9-26009fda6af0 nodeName:}" failed. No retries permitted until 2026-07-06 23:52:55.405356162 +0000 UTC m=+276.470688117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a778d1f2-e282-45bb-a9f9-26009fda6af0-tls-certs") pod "kube-auth-proxy-6c4984cb9-8bzw4" (UID: "a778d1f2-e282-45bb-a9f9-26009fda6af0") : secret "kube-auth-proxy-tls" not found
Jul 06 23:52:54.907669 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.907650    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a778d1f2-e282-45bb-a9f9-26009fda6af0-tmp\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:54.920502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:54.920478    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbrk4\" (UniqueName: \"kubernetes.io/projected/a778d1f2-e282-45bb-a9f9-26009fda6af0-kube-api-access-jbrk4\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:55.085952 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.085862    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"]
Jul 06 23:52:55.088652 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.088630    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.091240 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.091213    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 06 23:52:55.091350 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.091281    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 06 23:52:55.091888 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.091871    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:52:55.092036 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.091894    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 06 23:52:55.092289 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.091966    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 23:52:55.092814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.092378    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h5lwl\""
Jul 06 23:52:55.102578 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.102549    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"]
Jul 06 23:52:55.106738 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.106709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-metrics-cert\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.106848 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.106755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-cert\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.106848 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.106833    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dts68\" (UniqueName: \"kubernetes.io/projected/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-kube-api-access-dts68\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.106918 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.106906    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-manager-config\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.208357 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.208322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-manager-config\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.208548 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.208376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-metrics-cert\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.208548 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.208395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-cert\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.208548 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.208478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dts68\" (UniqueName: \"kubernetes.io/projected/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-kube-api-access-dts68\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.208949 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.208928    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-manager-config\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.211097 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.211070    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-metrics-cert\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.211205 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.211070    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-cert\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.217020 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.216986    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dts68\" (UniqueName: \"kubernetes.io/projected/d14a6c33-8f44-4f25-9a8f-0d3d71fb5898-kube-api-access-dts68\") pod \"lws-controller-manager-66459bdc77-bsgkb\" (UID: \"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898\") " pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.399358 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.399280    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:55.410281 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.410257    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a778d1f2-e282-45bb-a9f9-26009fda6af0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:55.413008 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.412976    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a778d1f2-e282-45bb-a9f9-26009fda6af0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8bzw4\" (UID: \"a778d1f2-e282-45bb-a9f9-26009fda6af0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:55.564083 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.564063    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"]
Jul 06 23:52:55.566821 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:52:55.566796    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14a6c33_8f44_4f25_9a8f_0d3d71fb5898.slice/crio-13ac485cfc52248f213c0d0346c7303adde5c839f630af922302e39dafa4c51b WatchSource:0}: Error finding container 13ac485cfc52248f213c0d0346c7303adde5c839f630af922302e39dafa4c51b: Status 404 returned error can't find the container with id 13ac485cfc52248f213c0d0346c7303adde5c839f630af922302e39dafa4c51b
Jul 06 23:52:55.668885 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.668789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"
Jul 06 23:52:55.809688 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:55.809647    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4"]
Jul 06 23:52:55.812798 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:52:55.812773    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda778d1f2_e282_45bb_a9f9_26009fda6af0.slice/crio-102de4e04779a9d65c47400666d4d9b6e151830ab9a70e6a2d2e38eb9c48448e WatchSource:0}: Error finding container 102de4e04779a9d65c47400666d4d9b6e151830ab9a70e6a2d2e38eb9c48448e: Status 404 returned error can't find the container with id 102de4e04779a9d65c47400666d4d9b6e151830ab9a70e6a2d2e38eb9c48448e
Jul 06 23:52:56.554529 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:56.554492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4" event={"ID":"a778d1f2-e282-45bb-a9f9-26009fda6af0","Type":"ContainerStarted","Data":"102de4e04779a9d65c47400666d4d9b6e151830ab9a70e6a2d2e38eb9c48448e"}
Jul 06 23:52:56.556194 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:56.556162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb" event={"ID":"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898","Type":"ContainerStarted","Data":"13ac485cfc52248f213c0d0346c7303adde5c839f630af922302e39dafa4c51b"}
Jul 06 23:52:59.569198 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:59.569165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4" event={"ID":"a778d1f2-e282-45bb-a9f9-26009fda6af0","Type":"ContainerStarted","Data":"55db56cfe7e452046eb343570faf0326eb2d0d7c48ecc0e509059af6aab69c11"}
Jul 06 23:52:59.570716 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:59.570693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb" event={"ID":"d14a6c33-8f44-4f25-9a8f-0d3d71fb5898","Type":"ContainerStarted","Data":"db20ce7e17735bb68d43be62c24d0b88f94c5f0fc2716bac0d98caca61d72ebf"}
Jul 06 23:52:59.570852 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:59.570781    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:52:59.585299 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:59.585251    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8bzw4" podStartSLOduration=2.674144247 podStartE2EDuration="5.585236768s" totalImagesPullingTime="2.911092521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:59.584467747 +0000 UTC m=+280.649799725" watchObservedRunningTime="2026-07-06 23:52:59.585236768 +0000 UTC m=+280.650568745"
Jul 06 23:52:59.601561 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:52:59.601513    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb" podStartSLOduration=1.4458830329999999 podStartE2EDuration="4.601496339s" totalImagesPullingTime="3.155613306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:59.600026512 +0000 UTC m=+280.665358484" watchObservedRunningTime="2026-07-06 23:52:59.601496339 +0000 UTC m=+280.666828315"
Jul 06 23:53:10.577699 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:53:10.577670    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66459bdc77-bsgkb"
Jul 06 23:53:19.263764 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:53:19.263732    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 06 23:53:19.266493 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:53:19.266473    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 06 23:53:19.266707 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:53:19.266693    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 23:54:34.720835 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.720792    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"]
Jul 06 23:54:34.724029 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.724007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:34.726761 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.726732    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 23:54:34.726907 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.726784    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pp4gb\""
Jul 06 23:54:34.726907 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.726836    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 23:54:34.731809 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.731783    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"]
Jul 06 23:54:34.831562 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.831509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hvvv\" (UniqueName: \"kubernetes.io/projected/72a99e57-e647-4837-8e3a-63f94438055f-kube-api-access-6hvvv\") pod \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" (UID: \"72a99e57-e647-4837-8e3a-63f94438055f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:34.932869 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.932833    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hvvv\" (UniqueName: \"kubernetes.io/projected/72a99e57-e647-4837-8e3a-63f94438055f-kube-api-access-6hvvv\") pod \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" (UID: \"72a99e57-e647-4837-8e3a-63f94438055f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:34.941533 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:34.941509    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hvvv\" (UniqueName: \"kubernetes.io/projected/72a99e57-e647-4837-8e3a-63f94438055f-kube-api-access-6hvvv\") pod \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" (UID: \"72a99e57-e647-4837-8e3a-63f94438055f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:35.034523 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:35.034408    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:35.179311 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:35.179278    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"]
Jul 06 23:54:35.182328 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:54:35.182299    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a99e57_e647_4837_8e3a_63f94438055f.slice/crio-3e2ececdf75dce2f44ba753bed1f7cc436faacfcf89042ac30e5054fb746495a WatchSource:0}: Error finding container 3e2ececdf75dce2f44ba753bed1f7cc436faacfcf89042ac30e5054fb746495a: Status 404 returned error can't find the container with id 3e2ececdf75dce2f44ba753bed1f7cc436faacfcf89042ac30e5054fb746495a
Jul 06 23:54:35.184336 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:35.184321    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:54:35.922648 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:35.922612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn" event={"ID":"72a99e57-e647-4837-8e3a-63f94438055f","Type":"ContainerStarted","Data":"3e2ececdf75dce2f44ba753bed1f7cc436faacfcf89042ac30e5054fb746495a"}
Jul 06 23:54:37.932645 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:37.932610    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn" event={"ID":"72a99e57-e647-4837-8e3a-63f94438055f","Type":"ContainerStarted","Data":"d14c4321c069c5e73e1b7dfdc4cd76dfde8af925b87ed56d27d984cc91485620"}
Jul 06 23:54:37.933131 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:37.932745    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:37.951960 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:37.951907    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn" podStartSLOduration=1.945256767 podStartE2EDuration="3.951891942s" totalImagesPullingTime="2.006635175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:37.950841159 +0000 UTC m=+379.016173136" watchObservedRunningTime="2026-07-06 23:54:37.951891942 +0000 UTC m=+379.017223919"
Jul 06 23:54:45.848470 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:45.848436    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"]
Jul 06 23:54:45.848917 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:45.848705    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn" podUID="72a99e57-e647-4837-8e3a-63f94438055f" containerName="manager" containerID="cri-o://d14c4321c069c5e73e1b7dfdc4cd76dfde8af925b87ed56d27d984cc91485620" gracePeriod=2
Jul 06 23:54:45.850795 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:45.850763    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:45.873957 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:45.873928    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"]
Jul 06 23:54:45.878035 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:54:45.877953    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"72a99e57-e647-4837-8e3a-63f94438055f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-7z8mn\": pods \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:45.880258 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:54:45.880228    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" is forbidden: User \"system:node:ip-10-0-130-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-92.ec2.internal' and this object" podUID="72a99e57-e647-4837-8e3a-63f94438055f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:45.966360 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:45.966328    2564 generic.go:356] "Generic (PLEG): container finished" podID="72a99e57-e647-4837-8e3a-63f94438055f" containerID="d14c4321c069c5e73e1b7dfdc4cd76dfde8af925b87ed56d27d984cc91485620" exitCode=0
Jul 06 23:54:46.008784 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.008749    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"]
Jul 06 23:54:46.009037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.009025    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a99e57-e647-4837-8e3a-63f94438055f" containerName="manager"
Jul 06 23:54:46.009082 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.009039    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a99e57-e647-4837-8e3a-63f94438055f" containerName="manager"
Jul 06 23:54:46.009121 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.009089    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a99e57-e647-4837-8e3a-63f94438055f" containerName="manager"
Jul 06 23:54:46.012136 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.012120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.016032 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.016006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ccst\" (UniqueName: \"kubernetes.io/projected/a0c565b2-a749-4742-a5a7-bd89149f617b-kube-api-access-6ccst\") pod \"kuadrant-operator-controller-manager-86878c769f-d9j7q\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.016134 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.016117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0c565b2-a749-4742-a5a7-bd89149f617b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d9j7q\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.016201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.016187    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mh67n\""
Jul 06 23:54:46.031739 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.031714    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"]
Jul 06 23:54:46.051199 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:54:46.051161    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" is forbidden: User \"system:node:ip-10-0-130-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-92.ec2.internal' and this object" podUID="72a99e57-e647-4837-8e3a-63f94438055f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:46.097944 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.097922    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:46.100257 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:54:46.100204    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" is forbidden: User \"system:node:ip-10-0-130-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-92.ec2.internal' and this object" podUID="72a99e57-e647-4837-8e3a-63f94438055f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:46.116622 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.116602    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccst\" (UniqueName: \"kubernetes.io/projected/a0c565b2-a749-4742-a5a7-bd89149f617b-kube-api-access-6ccst\") pod \"kuadrant-operator-controller-manager-86878c769f-d9j7q\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.116705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.116659    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0c565b2-a749-4742-a5a7-bd89149f617b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d9j7q\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.116953 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.116939    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0c565b2-a749-4742-a5a7-bd89149f617b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d9j7q\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.126446 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.126408    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ccst\" (UniqueName: \"kubernetes.io/projected/a0c565b2-a749-4742-a5a7-bd89149f617b-kube-api-access-6ccst\") pod \"kuadrant-operator-controller-manager-86878c769f-d9j7q\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.217178 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.217150    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hvvv\" (UniqueName: \"kubernetes.io/projected/72a99e57-e647-4837-8e3a-63f94438055f-kube-api-access-6hvvv\") pod \"72a99e57-e647-4837-8e3a-63f94438055f\" (UID: \"72a99e57-e647-4837-8e3a-63f94438055f\") "
Jul 06 23:54:46.219491 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.219464    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a99e57-e647-4837-8e3a-63f94438055f-kube-api-access-6hvvv" (OuterVolumeSpecName: "kube-api-access-6hvvv") pod "72a99e57-e647-4837-8e3a-63f94438055f" (UID: "72a99e57-e647-4837-8e3a-63f94438055f"). InnerVolumeSpecName "kube-api-access-6hvvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:54:46.318017 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.317982    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hvvv\" (UniqueName: \"kubernetes.io/projected/72a99e57-e647-4837-8e3a-63f94438055f-kube-api-access-6hvvv\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:54:46.324415 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.324388    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:46.471779 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.471749    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"]
Jul 06 23:54:46.475464 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:54:46.475439    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c565b2_a749_4742_a5a7_bd89149f617b.slice/crio-d0699df29336f010c51b95ad3c1a1ced757675fa3d434d6b3be610b226a4d246 WatchSource:0}: Error finding container d0699df29336f010c51b95ad3c1a1ced757675fa3d434d6b3be610b226a4d246: Status 404 returned error can't find the container with id d0699df29336f010c51b95ad3c1a1ced757675fa3d434d6b3be610b226a4d246
Jul 06 23:54:46.972188 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.972158    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:46.972660 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.972164    2564 scope.go:117] "RemoveContainer" containerID="d14c4321c069c5e73e1b7dfdc4cd76dfde8af925b87ed56d27d984cc91485620"
Jul 06 23:54:46.974634 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:46.974606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q" event={"ID":"a0c565b2-a749-4742-a5a7-bd89149f617b","Type":"ContainerStarted","Data":"d0699df29336f010c51b95ad3c1a1ced757675fa3d434d6b3be610b226a4d246"}
Jul 06 23:54:46.998002 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:54:46.997962    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" is forbidden: User \"system:node:ip-10-0-130-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-92.ec2.internal' and this object" podUID="72a99e57-e647-4837-8e3a-63f94438055f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:47.000564 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:54:47.000534    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7z8mn\" is forbidden: User \"system:node:ip-10-0-130-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-92.ec2.internal' and this object" podUID="72a99e57-e647-4837-8e3a-63f94438055f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7z8mn"
Jul 06 23:54:47.391218 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:47.391143    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a99e57-e647-4837-8e3a-63f94438055f" path="/var/lib/kubelet/pods/72a99e57-e647-4837-8e3a-63f94438055f/volumes"
Jul 06 23:54:49.656093 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.656057    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5ddc7dcb8f-mgh8k"]
Jul 06 23:54:49.659267 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.659248    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.662819 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.662794    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 23:54:49.663252 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.663232    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 23:54:49.663358 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.663298    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 23:54:49.663886 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.663871    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 23:54:49.663995 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.663976    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 23:54:49.664051 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.664028    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 23:54:49.664103 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.664085    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s7hfr\""
Jul 06 23:54:49.664155 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.664102    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 23:54:49.673058 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.673031    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 23:54:49.678472 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.678450    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ddc7dcb8f-mgh8k"]
Jul 06 23:54:49.747808 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.747771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-trusted-ca-bundle\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.747808 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.747820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-oauth-config\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.748062 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.747851    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-oauth-serving-cert\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.748062 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.748016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-serving-cert\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.748062 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.748056    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-service-ca\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.748222 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.748086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-config\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.748222 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.748113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt98r\" (UniqueName: \"kubernetes.io/projected/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-kube-api-access-tt98r\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.848502 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.848472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-oauth-serving-cert\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.848698 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.848568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-serving-cert\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.848698 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.848597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-service-ca\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.848698 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.848626    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-config\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.848698 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.848651    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt98r\" (UniqueName: \"kubernetes.io/projected/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-kube-api-access-tt98r\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.848915 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.848731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-trusted-ca-bundle\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.848915 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.848793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-oauth-config\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.849476 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.849452    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-oauth-serving-cert\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.849585 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.849521    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-service-ca\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.849654 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.849613    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-trusted-ca-bundle\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.849788 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.849769    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-config\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.851332 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.851308    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-serving-cert\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.851571 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.851552    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-console-oauth-config\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.858279 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.858251    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt98r\" (UniqueName: \"kubernetes.io/projected/57af95ba-10a3-46b4-9c56-c9cfd4c7b050-kube-api-access-tt98r\") pod \"console-5ddc7dcb8f-mgh8k\" (UID: \"57af95ba-10a3-46b4-9c56-c9cfd4c7b050\") " pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:49.968697 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:49.968650    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:51.114462 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:51.114414    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ddc7dcb8f-mgh8k"]
Jul 06 23:54:51.118013 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:54:51.117979    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57af95ba_10a3_46b4_9c56_c9cfd4c7b050.slice/crio-71b6923e08ff641f96f99e0f52ebfc0c5f8b6c84a32f30b06065a6948dd3fc5f WatchSource:0}: Error finding container 71b6923e08ff641f96f99e0f52ebfc0c5f8b6c84a32f30b06065a6948dd3fc5f: Status 404 returned error can't find the container with id 71b6923e08ff641f96f99e0f52ebfc0c5f8b6c84a32f30b06065a6948dd3fc5f
Jul 06 23:54:51.999870 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:51.999830    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q" event={"ID":"a0c565b2-a749-4742-a5a7-bd89149f617b","Type":"ContainerStarted","Data":"2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d"}
Jul 06 23:54:52.000044 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:51.999947    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:54:52.001418 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:52.001397    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ddc7dcb8f-mgh8k" event={"ID":"57af95ba-10a3-46b4-9c56-c9cfd4c7b050","Type":"ContainerStarted","Data":"56d8b586231670d1308a7b9bb1a6e16779053c443738c70e0b818885380b32df"}
Jul 06 23:54:52.001572 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:52.001449    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ddc7dcb8f-mgh8k" event={"ID":"57af95ba-10a3-46b4-9c56-c9cfd4c7b050","Type":"ContainerStarted","Data":"71b6923e08ff641f96f99e0f52ebfc0c5f8b6c84a32f30b06065a6948dd3fc5f"}
Jul 06 23:54:52.023985 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:52.023923    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q" podStartSLOduration=2.460460476 podStartE2EDuration="7.023908921s" totalImagesPullingTime="4.563448445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:52.021522161 +0000 UTC m=+393.086854139" watchObservedRunningTime="2026-07-06 23:54:52.023908921 +0000 UTC m=+393.089240898"
Jul 06 23:54:52.041208 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:52.041160    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5ddc7dcb8f-mgh8k" podStartSLOduration=3.041144791 podStartE2EDuration="3.041144791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:52.039312984 +0000 UTC m=+393.104644962" watchObservedRunningTime="2026-07-06 23:54:52.041144791 +0000 UTC m=+393.106476767"
Jul 06 23:54:59.969069 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:59.969031    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:59.969487 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:59.969092    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:54:59.973975 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:54:59.973947    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:55:00.043332 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:00.043297    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5ddc7dcb8f-mgh8k"
Jul 06 23:55:03.008033 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:03.008004    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 06 23:55:21.475534 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.475503    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"]
Jul 06 23:55:21.481042 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.481023    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.483784 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.483764    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 23:55:21.483784 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.483775    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cxzj5\""
Jul 06 23:55:21.490184 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.490158    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"]
Jul 06 23:55:21.574095 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.574058    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"]
Jul 06 23:55:21.622973 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.622932    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6q9w\" (UniqueName: \"kubernetes.io/projected/e7402d88-200c-4117-a159-d1275b0ecf8e-kube-api-access-n6q9w\") pod \"limitador-limitador-64c94fd64f-hlxpx\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.623152 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.623038    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7402d88-200c-4117-a159-d1275b0ecf8e-config-file\") pod \"limitador-limitador-64c94fd64f-hlxpx\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.723688 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.723644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6q9w\" (UniqueName: \"kubernetes.io/projected/e7402d88-200c-4117-a159-d1275b0ecf8e-kube-api-access-n6q9w\") pod \"limitador-limitador-64c94fd64f-hlxpx\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.723866 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.723749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7402d88-200c-4117-a159-d1275b0ecf8e-config-file\") pod \"limitador-limitador-64c94fd64f-hlxpx\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.724345 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.724324    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7402d88-200c-4117-a159-d1275b0ecf8e-config-file\") pod \"limitador-limitador-64c94fd64f-hlxpx\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.734047 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.733976    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6q9w\" (UniqueName: \"kubernetes.io/projected/e7402d88-200c-4117-a159-d1275b0ecf8e-kube-api-access-n6q9w\") pod \"limitador-limitador-64c94fd64f-hlxpx\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.792046 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.792012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:21.927696 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:21.927666    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"]
Jul 06 23:55:21.930477 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:55:21.930452    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7402d88_200c_4117_a159_d1275b0ecf8e.slice/crio-58c73a0e62680107ca28efd637d47bbde1026642ef3845264441a7357219cd31 WatchSource:0}: Error finding container 58c73a0e62680107ca28efd637d47bbde1026642ef3845264441a7357219cd31: Status 404 returned error can't find the container with id 58c73a0e62680107ca28efd637d47bbde1026642ef3845264441a7357219cd31
Jul 06 23:55:22.130122 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.130033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx" event={"ID":"e7402d88-200c-4117-a159-d1275b0ecf8e","Type":"ContainerStarted","Data":"58c73a0e62680107ca28efd637d47bbde1026642ef3845264441a7357219cd31"}
Jul 06 23:55:22.357766 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.357724    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-wf29m"]
Jul 06 23:55:22.389581 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.389503    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wf29m"]
Jul 06 23:55:22.389711 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.389610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wf29m"
Jul 06 23:55:22.392798 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.392777    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ktbzk\""
Jul 06 23:55:22.530318 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.530288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th89d\" (UniqueName: \"kubernetes.io/projected/3edeae9e-025b-402a-beb7-4428973d8434-kube-api-access-th89d\") pod \"authorino-58cd955675-wf29m\" (UID: \"3edeae9e-025b-402a-beb7-4428973d8434\") " pod="kuadrant-system/authorino-58cd955675-wf29m"
Jul 06 23:55:22.631200 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.631162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th89d\" (UniqueName: \"kubernetes.io/projected/3edeae9e-025b-402a-beb7-4428973d8434-kube-api-access-th89d\") pod \"authorino-58cd955675-wf29m\" (UID: \"3edeae9e-025b-402a-beb7-4428973d8434\") " pod="kuadrant-system/authorino-58cd955675-wf29m"
Jul 06 23:55:22.646068 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.645996    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th89d\" (UniqueName: \"kubernetes.io/projected/3edeae9e-025b-402a-beb7-4428973d8434-kube-api-access-th89d\") pod \"authorino-58cd955675-wf29m\" (UID: \"3edeae9e-025b-402a-beb7-4428973d8434\") " pod="kuadrant-system/authorino-58cd955675-wf29m"
Jul 06 23:55:22.698882 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.698832    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wf29m"
Jul 06 23:55:22.879298 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:22.874859    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wf29m"]
Jul 06 23:55:22.879298 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:55:22.878225    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3edeae9e_025b_402a_beb7_4428973d8434.slice/crio-61baedb17b43db3d097c08da31c103961afa9b170177be0ff1ed3835793cb0d3 WatchSource:0}: Error finding container 61baedb17b43db3d097c08da31c103961afa9b170177be0ff1ed3835793cb0d3: Status 404 returned error can't find the container with id 61baedb17b43db3d097c08da31c103961afa9b170177be0ff1ed3835793cb0d3
Jul 06 23:55:23.136662 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:23.136630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wf29m" event={"ID":"3edeae9e-025b-402a-beb7-4428973d8434","Type":"ContainerStarted","Data":"61baedb17b43db3d097c08da31c103961afa9b170177be0ff1ed3835793cb0d3"}
Jul 06 23:55:27.161935 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:27.161893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wf29m" event={"ID":"3edeae9e-025b-402a-beb7-4428973d8434","Type":"ContainerStarted","Data":"599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc"}
Jul 06 23:55:27.163334 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:27.163314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx" event={"ID":"e7402d88-200c-4117-a159-d1275b0ecf8e","Type":"ContainerStarted","Data":"a735c6cd67f80804141d5265a18af6c38e6cb99a1f3ba337387abdeac603ae23"}
Jul 06 23:55:27.163413 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:27.163358    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:27.177179 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:27.177128    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-wf29m" podStartSLOduration=1.417893324 podStartE2EDuration="5.17711521s" totalImagesPullingTime="3.759221886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:27.176377007 +0000 UTC m=+428.241708985" watchObservedRunningTime="2026-07-06 23:55:27.17711521 +0000 UTC m=+428.242447164"
Jul 06 23:55:27.193495 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:27.193417    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx" podStartSLOduration=1.492880899 podStartE2EDuration="6.193402507s" totalImagesPullingTime="4.700521608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:27.192182575 +0000 UTC m=+428.257514553" watchObservedRunningTime="2026-07-06 23:55:27.193402507 +0000 UTC m=+428.258734484"
Jul 06 23:55:29.440008 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:29.439971    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wf29m"]
Jul 06 23:55:29.440458 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:29.440140    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-wf29m" podUID="3edeae9e-025b-402a-beb7-4428973d8434" containerName="authorino" containerID="cri-o://599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc" gracePeriod=30
Jul 06 23:55:29.688034 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:29.688011    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wf29m"
Jul 06 23:55:29.799564 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:29.799533    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th89d\" (UniqueName: \"kubernetes.io/projected/3edeae9e-025b-402a-beb7-4428973d8434-kube-api-access-th89d\") pod \"3edeae9e-025b-402a-beb7-4428973d8434\" (UID: \"3edeae9e-025b-402a-beb7-4428973d8434\") "
Jul 06 23:55:29.801723 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:29.801700    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3edeae9e-025b-402a-beb7-4428973d8434-kube-api-access-th89d" (OuterVolumeSpecName: "kube-api-access-th89d") pod "3edeae9e-025b-402a-beb7-4428973d8434" (UID: "3edeae9e-025b-402a-beb7-4428973d8434"). InnerVolumeSpecName "kube-api-access-th89d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:55:29.900959 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:29.900925    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th89d\" (UniqueName: \"kubernetes.io/projected/3edeae9e-025b-402a-beb7-4428973d8434-kube-api-access-th89d\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:55:30.178404 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.178315    2564 generic.go:356] "Generic (PLEG): container finished" podID="3edeae9e-025b-402a-beb7-4428973d8434" containerID="599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc" exitCode=0
Jul 06 23:55:30.178404 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.178353    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wf29m"
Jul 06 23:55:30.178404 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.178366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wf29m" event={"ID":"3edeae9e-025b-402a-beb7-4428973d8434","Type":"ContainerDied","Data":"599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc"}
Jul 06 23:55:30.178404 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.178397    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wf29m" event={"ID":"3edeae9e-025b-402a-beb7-4428973d8434","Type":"ContainerDied","Data":"61baedb17b43db3d097c08da31c103961afa9b170177be0ff1ed3835793cb0d3"}
Jul 06 23:55:30.178679 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.178414    2564 scope.go:117] "RemoveContainer" containerID="599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc"
Jul 06 23:55:30.186683 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.186664    2564 scope.go:117] "RemoveContainer" containerID="599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc"
Jul 06 23:55:30.186987 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:55:30.186968    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc\": container with ID starting with 599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc not found: ID does not exist" containerID="599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc"
Jul 06 23:55:30.187037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.186999    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc"} err="failed to get container status \"599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc\": rpc error: code = NotFound desc = could not find container \"599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc\": container with ID starting with 599fe06f1cd65cb8cc8ed62e34253d4625234396b16400b0f9522ea053e9bcdc not found: ID does not exist"
Jul 06 23:55:30.203350 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.203319    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wf29m"]
Jul 06 23:55:30.209320 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:30.209295    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-wf29m"]
Jul 06 23:55:31.390145 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:31.390106    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3edeae9e-025b-402a-beb7-4428973d8434" path="/var/lib/kubelet/pods/3edeae9e-025b-402a-beb7-4428973d8434/volumes"
Jul 06 23:55:38.169492 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:38.169459    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:40.488234 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:40.488153    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"]
Jul 06 23:55:40.488690 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:40.488379    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx" podUID="e7402d88-200c-4117-a159-d1275b0ecf8e" containerName="limitador" containerID="cri-o://a735c6cd67f80804141d5265a18af6c38e6cb99a1f3ba337387abdeac603ae23" gracePeriod=30
Jul 06 23:55:41.226301 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.226268    2564 generic.go:356] "Generic (PLEG): container finished" podID="e7402d88-200c-4117-a159-d1275b0ecf8e" containerID="a735c6cd67f80804141d5265a18af6c38e6cb99a1f3ba337387abdeac603ae23" exitCode=0
Jul 06 23:55:41.226484 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.226339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx" event={"ID":"e7402d88-200c-4117-a159-d1275b0ecf8e","Type":"ContainerDied","Data":"a735c6cd67f80804141d5265a18af6c38e6cb99a1f3ba337387abdeac603ae23"}
Jul 06 23:55:41.434804 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.434782    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:41.490814 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.490740    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7402d88-200c-4117-a159-d1275b0ecf8e-config-file\") pod \"e7402d88-200c-4117-a159-d1275b0ecf8e\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") "
Jul 06 23:55:41.491158 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.490820    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6q9w\" (UniqueName: \"kubernetes.io/projected/e7402d88-200c-4117-a159-d1275b0ecf8e-kube-api-access-n6q9w\") pod \"e7402d88-200c-4117-a159-d1275b0ecf8e\" (UID: \"e7402d88-200c-4117-a159-d1275b0ecf8e\") "
Jul 06 23:55:41.491158 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.491068    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7402d88-200c-4117-a159-d1275b0ecf8e-config-file" (OuterVolumeSpecName: "config-file") pod "e7402d88-200c-4117-a159-d1275b0ecf8e" (UID: "e7402d88-200c-4117-a159-d1275b0ecf8e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:55:41.492977 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.492951    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7402d88-200c-4117-a159-d1275b0ecf8e-kube-api-access-n6q9w" (OuterVolumeSpecName: "kube-api-access-n6q9w") pod "e7402d88-200c-4117-a159-d1275b0ecf8e" (UID: "e7402d88-200c-4117-a159-d1275b0ecf8e"). InnerVolumeSpecName "kube-api-access-n6q9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:55:41.591498 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.591461    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6q9w\" (UniqueName: \"kubernetes.io/projected/e7402d88-200c-4117-a159-d1275b0ecf8e-kube-api-access-n6q9w\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:55:41.591498 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:41.591490    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7402d88-200c-4117-a159-d1275b0ecf8e-config-file\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:55:42.231821 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:42.231793    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"
Jul 06 23:55:42.231990 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:42.231784    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hlxpx" event={"ID":"e7402d88-200c-4117-a159-d1275b0ecf8e","Type":"ContainerDied","Data":"58c73a0e62680107ca28efd637d47bbde1026642ef3845264441a7357219cd31"}
Jul 06 23:55:42.231990 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:42.231927    2564 scope.go:117] "RemoveContainer" containerID="a735c6cd67f80804141d5265a18af6c38e6cb99a1f3ba337387abdeac603ae23"
Jul 06 23:55:42.260272 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:42.260243    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"]
Jul 06 23:55:42.265574 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:42.265543    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hlxpx"]
Jul 06 23:55:43.007448 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.007399    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-c297r"]
Jul 06 23:55:43.007916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.007829    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3edeae9e-025b-402a-beb7-4428973d8434" containerName="authorino"
Jul 06 23:55:43.007916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.007846    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3edeae9e-025b-402a-beb7-4428973d8434" containerName="authorino"
Jul 06 23:55:43.007916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.007866    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7402d88-200c-4117-a159-d1275b0ecf8e" containerName="limitador"
Jul 06 23:55:43.007916 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.007875    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7402d88-200c-4117-a159-d1275b0ecf8e" containerName="limitador"
Jul 06 23:55:43.008114 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.007945    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7402d88-200c-4117-a159-d1275b0ecf8e" containerName="limitador"
Jul 06 23:55:43.008114 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.007957    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3edeae9e-025b-402a-beb7-4428973d8434" containerName="authorino"
Jul 06 23:55:43.012626 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.012603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.016110 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.016089    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 06 23:55:43.016229 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.016089    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-t44bg\""
Jul 06 23:55:43.030959 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.030915    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-c297r"]
Jul 06 23:55:43.105269 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.105225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e212522e-8b82-4d91-811f-387210396974-data\") pod \"postgres-69fb4594cf-c297r\" (UID: \"e212522e-8b82-4d91-811f-387210396974\") " pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.105269 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.105266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsdvs\" (UniqueName: \"kubernetes.io/projected/e212522e-8b82-4d91-811f-387210396974-kube-api-access-gsdvs\") pod \"postgres-69fb4594cf-c297r\" (UID: \"e212522e-8b82-4d91-811f-387210396974\") " pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.206606 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.206567    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e212522e-8b82-4d91-811f-387210396974-data\") pod \"postgres-69fb4594cf-c297r\" (UID: \"e212522e-8b82-4d91-811f-387210396974\") " pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.206606 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.206604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsdvs\" (UniqueName: \"kubernetes.io/projected/e212522e-8b82-4d91-811f-387210396974-kube-api-access-gsdvs\") pod \"postgres-69fb4594cf-c297r\" (UID: \"e212522e-8b82-4d91-811f-387210396974\") " pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.207029 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.207005    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e212522e-8b82-4d91-811f-387210396974-data\") pod \"postgres-69fb4594cf-c297r\" (UID: \"e212522e-8b82-4d91-811f-387210396974\") " pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.216018 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.215990    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsdvs\" (UniqueName: \"kubernetes.io/projected/e212522e-8b82-4d91-811f-387210396974-kube-api-access-gsdvs\") pod \"postgres-69fb4594cf-c297r\" (UID: \"e212522e-8b82-4d91-811f-387210396974\") " pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.322555 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.322471    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:43.390980 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.390846    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7402d88-200c-4117-a159-d1275b0ecf8e" path="/var/lib/kubelet/pods/e7402d88-200c-4117-a159-d1275b0ecf8e/volumes"
Jul 06 23:55:43.459513 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:43.459469    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-c297r"]
Jul 06 23:55:43.462289 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:55:43.462262    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode212522e_8b82_4d91_811f_387210396974.slice/crio-2c1ca7aef98622c4692b1dd9968db9b5e47f123e35e87dd5d21e28d589a6c1f9 WatchSource:0}: Error finding container 2c1ca7aef98622c4692b1dd9968db9b5e47f123e35e87dd5d21e28d589a6c1f9: Status 404 returned error can't find the container with id 2c1ca7aef98622c4692b1dd9968db9b5e47f123e35e87dd5d21e28d589a6c1f9
Jul 06 23:55:44.243579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:44.243539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-c297r" event={"ID":"e212522e-8b82-4d91-811f-387210396974","Type":"ContainerStarted","Data":"2c1ca7aef98622c4692b1dd9968db9b5e47f123e35e87dd5d21e28d589a6c1f9"}
Jul 06 23:55:49.269682 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:49.269643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-c297r" event={"ID":"e212522e-8b82-4d91-811f-387210396974","Type":"ContainerStarted","Data":"7c3e5e0809ff6f605350ec8c42764d78f86662b50df5e5d2c76a72d6cd346d77"}
Jul 06 23:55:49.270165 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:49.269732    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:55:49.309222 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:49.309166    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-c297r" podStartSLOduration=2.311874903 podStartE2EDuration="7.309149165s" totalImagesPullingTime="4.997274262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:49.307994581 +0000 UTC m=+450.373326560" watchObservedRunningTime="2026-07-06 23:55:49.309149165 +0000 UTC m=+450.374481196"
Jul 06 23:55:55.304062 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:55:55.304030    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-c297r"
Jul 06 23:56:02.202433 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.202394    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5669f446d5-h4nwq"]
Jul 06 23:56:02.208456 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.208411    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.211925 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.211900    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 23:56:02.211925 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.211951    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 23:56:02.212219 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.212134    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bhtjd\""
Jul 06 23:56:02.220969 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.220946    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5669f446d5-h4nwq"]
Jul 06 23:56:02.378931 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.378890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eda5e719-437a-444e-9bf1-096a25e76c02-webhook-cert\") pod \"maas-controller-5669f446d5-h4nwq\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") " pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.379108 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.378998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76lcq\" (UniqueName: \"kubernetes.io/projected/eda5e719-437a-444e-9bf1-096a25e76c02-kube-api-access-76lcq\") pod \"maas-controller-5669f446d5-h4nwq\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") " pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.480259 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.480228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76lcq\" (UniqueName: \"kubernetes.io/projected/eda5e719-437a-444e-9bf1-096a25e76c02-kube-api-access-76lcq\") pod \"maas-controller-5669f446d5-h4nwq\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") " pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.480516 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.480293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eda5e719-437a-444e-9bf1-096a25e76c02-webhook-cert\") pod \"maas-controller-5669f446d5-h4nwq\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") " pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.482813 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.482787    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eda5e719-437a-444e-9bf1-096a25e76c02-webhook-cert\") pod \"maas-controller-5669f446d5-h4nwq\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") " pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.488849 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.488827    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76lcq\" (UniqueName: \"kubernetes.io/projected/eda5e719-437a-444e-9bf1-096a25e76c02-kube-api-access-76lcq\") pod \"maas-controller-5669f446d5-h4nwq\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") " pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.519491 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.519458    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:02.655800 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:02.655776    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5669f446d5-h4nwq"]
Jul 06 23:56:02.658078 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:56:02.658043    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda5e719_437a_444e_9bf1_096a25e76c02.slice/crio-9eb20974c10ed14a90030f46ff2cead274de0871718c080036093f5bfb71cb09 WatchSource:0}: Error finding container 9eb20974c10ed14a90030f46ff2cead274de0871718c080036093f5bfb71cb09: Status 404 returned error can't find the container with id 9eb20974c10ed14a90030f46ff2cead274de0871718c080036093f5bfb71cb09
Jul 06 23:56:03.332479 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:03.332446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5669f446d5-h4nwq" event={"ID":"eda5e719-437a-444e-9bf1-096a25e76c02","Type":"ContainerStarted","Data":"9eb20974c10ed14a90030f46ff2cead274de0871718c080036093f5bfb71cb09"}
Jul 06 23:56:05.343496 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:05.343465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5669f446d5-h4nwq" event={"ID":"eda5e719-437a-444e-9bf1-096a25e76c02","Type":"ContainerStarted","Data":"e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4"}
Jul 06 23:56:05.343884 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:05.343533    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:05.373716 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:05.373655    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5669f446d5-h4nwq" podStartSLOduration=1.6039541659999998 podStartE2EDuration="3.373639464s" totalImagesPullingTime="1.769685298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:05.372718844 +0000 UTC m=+466.438050823" watchObservedRunningTime="2026-07-06 23:56:05.373639464 +0000 UTC m=+466.438971441"
Jul 06 23:56:16.354717 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.354687    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:16.454119 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.454085    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5669f446d5-h4nwq"]
Jul 06 23:56:16.454293 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.454273    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5669f446d5-h4nwq" podUID="eda5e719-437a-444e-9bf1-096a25e76c02" containerName="manager" containerID="cri-o://e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4" gracePeriod=30
Jul 06 23:56:16.697468 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.697442    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:16.703486 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.703462    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eda5e719-437a-444e-9bf1-096a25e76c02-webhook-cert\") pod \"eda5e719-437a-444e-9bf1-096a25e76c02\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") "
Jul 06 23:56:16.703624 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.703495    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76lcq\" (UniqueName: \"kubernetes.io/projected/eda5e719-437a-444e-9bf1-096a25e76c02-kube-api-access-76lcq\") pod \"eda5e719-437a-444e-9bf1-096a25e76c02\" (UID: \"eda5e719-437a-444e-9bf1-096a25e76c02\") "
Jul 06 23:56:16.705889 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.705861    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda5e719-437a-444e-9bf1-096a25e76c02-kube-api-access-76lcq" (OuterVolumeSpecName: "kube-api-access-76lcq") pod "eda5e719-437a-444e-9bf1-096a25e76c02" (UID: "eda5e719-437a-444e-9bf1-096a25e76c02"). InnerVolumeSpecName "kube-api-access-76lcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:56:16.705889 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.705870    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eda5e719-437a-444e-9bf1-096a25e76c02-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eda5e719-437a-444e-9bf1-096a25e76c02" (UID: "eda5e719-437a-444e-9bf1-096a25e76c02"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:56:16.804238 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.804197    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eda5e719-437a-444e-9bf1-096a25e76c02-webhook-cert\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:56:16.804238 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:16.804232    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76lcq\" (UniqueName: \"kubernetes.io/projected/eda5e719-437a-444e-9bf1-096a25e76c02-kube-api-access-76lcq\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 06 23:56:17.396918 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.396885    2564 generic.go:356] "Generic (PLEG): container finished" podID="eda5e719-437a-444e-9bf1-096a25e76c02" containerID="e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4" exitCode=0
Jul 06 23:56:17.396918 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.396924    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5669f446d5-h4nwq"
Jul 06 23:56:17.397532 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.396972    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5669f446d5-h4nwq" event={"ID":"eda5e719-437a-444e-9bf1-096a25e76c02","Type":"ContainerDied","Data":"e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4"}
Jul 06 23:56:17.397532 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.397029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5669f446d5-h4nwq" event={"ID":"eda5e719-437a-444e-9bf1-096a25e76c02","Type":"ContainerDied","Data":"9eb20974c10ed14a90030f46ff2cead274de0871718c080036093f5bfb71cb09"}
Jul 06 23:56:17.397532 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.397061    2564 scope.go:117] "RemoveContainer" containerID="e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4"
Jul 06 23:56:17.406307 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.406286    2564 scope.go:117] "RemoveContainer" containerID="e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4"
Jul 06 23:56:17.406742 ip-10-0-130-92 kubenswrapper[2564]: E0706 23:56:17.406718    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4\": container with ID starting with e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4 not found: ID does not exist" containerID="e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4"
Jul 06 23:56:17.406801 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.406754    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4"} err="failed to get container status \"e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4\": rpc error: code = NotFound desc = could not find container \"e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4\": container with ID starting with e611bb8e54df806bfbd112ad79d13b66439c31f9e5bb50be790ee822b93b15a4 not found: ID does not exist"
Jul 06 23:56:17.416864 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.416834    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5669f446d5-h4nwq"]
Jul 06 23:56:17.421201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:17.421173    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5669f446d5-h4nwq"]
Jul 06 23:56:19.390919 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:19.390879    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eda5e719-437a-444e-9bf1-096a25e76c02" path="/var/lib/kubelet/pods/eda5e719-437a-444e-9bf1-096a25e76c02/volumes"
Jul 06 23:56:38.120113 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.120077    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7777c48764-pj5gk"]
Jul 06 23:56:38.120589 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.120401    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eda5e719-437a-444e-9bf1-096a25e76c02" containerName="manager"
Jul 06 23:56:38.120589 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.120411    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eda5e719-437a-444e-9bf1-096a25e76c02" containerName="manager"
Jul 06 23:56:38.120589 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.120484    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eda5e719-437a-444e-9bf1-096a25e76c02" containerName="manager"
Jul 06 23:56:38.128998 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.128972    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.131637 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.131613    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-7qrr6\""
Jul 06 23:56:38.131983 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.131967    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 06 23:56:38.134994 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.134968    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7777c48764-pj5gk"]
Jul 06 23:56:38.173247 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.173212    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1965738a-61fc-4bab-b90d-a5d34a6fe670-openshift-service-ca\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.173469 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.173265    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1965738a-61fc-4bab-b90d-a5d34a6fe670-maas-api-tls\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.173469 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.173286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkqdh\" (UniqueName: \"kubernetes.io/projected/1965738a-61fc-4bab-b90d-a5d34a6fe670-kube-api-access-tkqdh\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.273958 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.273920    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1965738a-61fc-4bab-b90d-a5d34a6fe670-maas-api-tls\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.273958 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.273962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkqdh\" (UniqueName: \"kubernetes.io/projected/1965738a-61fc-4bab-b90d-a5d34a6fe670-kube-api-access-tkqdh\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.274193 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.274012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1965738a-61fc-4bab-b90d-a5d34a6fe670-openshift-service-ca\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.274645 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.274627    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1965738a-61fc-4bab-b90d-a5d34a6fe670-openshift-service-ca\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.276617 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.276597    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1965738a-61fc-4bab-b90d-a5d34a6fe670-maas-api-tls\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.283493 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.283463    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkqdh\" (UniqueName: \"kubernetes.io/projected/1965738a-61fc-4bab-b90d-a5d34a6fe670-kube-api-access-tkqdh\") pod \"maas-api-7777c48764-pj5gk\" (UID: \"1965738a-61fc-4bab-b90d-a5d34a6fe670\") " pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.441238 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.441152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:38.592238 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.592208    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7777c48764-pj5gk"]
Jul 06 23:56:38.595171 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:56:38.595139    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1965738a_61fc_4bab_b90d_a5d34a6fe670.slice/crio-144de809106b0b3047e153da060aaac91b8a0d6ce58b911f555289074d5accaa WatchSource:0}: Error finding container 144de809106b0b3047e153da060aaac91b8a0d6ce58b911f555289074d5accaa: Status 404 returned error can't find the container with id 144de809106b0b3047e153da060aaac91b8a0d6ce58b911f555289074d5accaa
Jul 06 23:56:38.672212 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.672173    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-7jv49"]
Jul 06 23:56:38.675683 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.675664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:38.678413 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.678395    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 23:56:38.678413 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.678413    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qgplb\""
Jul 06 23:56:38.685206 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.685173    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-7jv49"]
Jul 06 23:56:38.705177 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.705089    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-lnvtg"]
Jul 06 23:56:38.707810 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.707792    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:38.718009 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.717983    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-lnvtg"]
Jul 06 23:56:38.778579 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.778534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljjbx\" (UniqueName: \"kubernetes.io/projected/feb71c59-3bbe-427b-b893-fe5312ca8497-kube-api-access-ljjbx\") pod \"payload-processing-58f86c7798-lnvtg\" (UID: \"feb71c59-3bbe-427b-b893-fe5312ca8497\") " pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:38.778762 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.778595    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0812424-e05f-4565-bfcc-ee3dd83e1bf3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-7jv49\" (UID: \"d0812424-e05f-4565-bfcc-ee3dd83e1bf3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:38.778762 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.778663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcnwr\" (UniqueName: \"kubernetes.io/projected/d0812424-e05f-4565-bfcc-ee3dd83e1bf3-kube-api-access-zcnwr\") pod \"payload-pre-processing-7964457696-7jv49\" (UID: \"d0812424-e05f-4565-bfcc-ee3dd83e1bf3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:38.778762 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.778728    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feb71c59-3bbe-427b-b893-fe5312ca8497-plugins-config-volume\") pod \"payload-processing-58f86c7798-lnvtg\" (UID: \"feb71c59-3bbe-427b-b893-fe5312ca8497\") " pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:38.879839 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.879802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljjbx\" (UniqueName: \"kubernetes.io/projected/feb71c59-3bbe-427b-b893-fe5312ca8497-kube-api-access-ljjbx\") pod \"payload-processing-58f86c7798-lnvtg\" (UID: \"feb71c59-3bbe-427b-b893-fe5312ca8497\") " pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:38.880059 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.879870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0812424-e05f-4565-bfcc-ee3dd83e1bf3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-7jv49\" (UID: \"d0812424-e05f-4565-bfcc-ee3dd83e1bf3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:38.880059 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.879902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcnwr\" (UniqueName: \"kubernetes.io/projected/d0812424-e05f-4565-bfcc-ee3dd83e1bf3-kube-api-access-zcnwr\") pod \"payload-pre-processing-7964457696-7jv49\" (UID: \"d0812424-e05f-4565-bfcc-ee3dd83e1bf3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:38.880059 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.879955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feb71c59-3bbe-427b-b893-fe5312ca8497-plugins-config-volume\") pod \"payload-processing-58f86c7798-lnvtg\" (UID: \"feb71c59-3bbe-427b-b893-fe5312ca8497\") " pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:38.880726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.880699    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feb71c59-3bbe-427b-b893-fe5312ca8497-plugins-config-volume\") pod \"payload-processing-58f86c7798-lnvtg\" (UID: \"feb71c59-3bbe-427b-b893-fe5312ca8497\") " pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:38.880828 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.880774    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0812424-e05f-4565-bfcc-ee3dd83e1bf3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-7jv49\" (UID: \"d0812424-e05f-4565-bfcc-ee3dd83e1bf3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:38.889734 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.889707    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljjbx\" (UniqueName: \"kubernetes.io/projected/feb71c59-3bbe-427b-b893-fe5312ca8497-kube-api-access-ljjbx\") pod \"payload-processing-58f86c7798-lnvtg\" (UID: \"feb71c59-3bbe-427b-b893-fe5312ca8497\") " pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:38.889859 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.889737    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcnwr\" (UniqueName: \"kubernetes.io/projected/d0812424-e05f-4565-bfcc-ee3dd83e1bf3-kube-api-access-zcnwr\") pod \"payload-pre-processing-7964457696-7jv49\" (UID: \"d0812424-e05f-4565-bfcc-ee3dd83e1bf3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:38.987507 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:38.987467    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:39.019283 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:39.019250    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:39.166363 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:39.166303    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-7jv49"]
Jul 06 23:56:39.167830 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:56:39.167796    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0812424_e05f_4565_bfcc_ee3dd83e1bf3.slice/crio-4683d94c7924615531190f45138cab82a2a877642a15d0513766278fefe6d42e WatchSource:0}: Error finding container 4683d94c7924615531190f45138cab82a2a877642a15d0513766278fefe6d42e: Status 404 returned error can't find the container with id 4683d94c7924615531190f45138cab82a2a877642a15d0513766278fefe6d42e
Jul 06 23:56:39.191831 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:39.191795    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-lnvtg"]
Jul 06 23:56:39.194739 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:56:39.194707    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeb71c59_3bbe_427b_b893_fe5312ca8497.slice/crio-a37eeb355a0bef2eb4fff17415983cf58f5e4795504e41fcfce594d751ab88f7 WatchSource:0}: Error finding container a37eeb355a0bef2eb4fff17415983cf58f5e4795504e41fcfce594d751ab88f7: Status 404 returned error can't find the container with id a37eeb355a0bef2eb4fff17415983cf58f5e4795504e41fcfce594d751ab88f7
Jul 06 23:56:39.488500 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:39.488464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7777c48764-pj5gk" event={"ID":"1965738a-61fc-4bab-b90d-a5d34a6fe670","Type":"ContainerStarted","Data":"144de809106b0b3047e153da060aaac91b8a0d6ce58b911f555289074d5accaa"}
Jul 06 23:56:39.489802 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:39.489767    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-lnvtg" event={"ID":"feb71c59-3bbe-427b-b893-fe5312ca8497","Type":"ContainerStarted","Data":"a37eeb355a0bef2eb4fff17415983cf58f5e4795504e41fcfce594d751ab88f7"}
Jul 06 23:56:39.491103 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:39.491065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-7jv49" event={"ID":"d0812424-e05f-4565-bfcc-ee3dd83e1bf3","Type":"ContainerStarted","Data":"4683d94c7924615531190f45138cab82a2a877642a15d0513766278fefe6d42e"}
Jul 06 23:56:41.504372 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.504338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-lnvtg" event={"ID":"feb71c59-3bbe-427b-b893-fe5312ca8497","Type":"ContainerStarted","Data":"3b6f9febf87f8d49da071f50fddee3800409795f141288f852cf179e83a40f48"}
Jul 06 23:56:41.504866 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.504469    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:41.505881 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.505856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-7jv49" event={"ID":"d0812424-e05f-4565-bfcc-ee3dd83e1bf3","Type":"ContainerStarted","Data":"33049e488fc6783da94f5a4aec6820fd75a9fdc51a53ec772bccb6b13d3640c2"}
Jul 06 23:56:41.505998 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.505925    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:41.507288 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.507268    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7777c48764-pj5gk" event={"ID":"1965738a-61fc-4bab-b90d-a5d34a6fe670","Type":"ContainerStarted","Data":"320cb4f9f89dddf3d3035d2e6ea244109497a982d5051f5ddcae182cc5186e5b"}
Jul 06 23:56:41.507394 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.507369    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:41.523139 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.523082    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-lnvtg" podStartSLOduration=2.248900939 podStartE2EDuration="3.52306115s" totalImagesPullingTime="1.274160211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:41.521286082 +0000 UTC m=+502.586618060" watchObservedRunningTime="2026-07-06 23:56:41.52306115 +0000 UTC m=+502.588393128"
Jul 06 23:56:41.540368 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.540316    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-7jv49" podStartSLOduration=2.235642162 podStartE2EDuration="3.540299936s" totalImagesPullingTime="1.304657774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:41.53844262 +0000 UTC m=+502.603774595" watchObservedRunningTime="2026-07-06 23:56:41.540299936 +0000 UTC m=+502.605631913"
Jul 06 23:56:41.557565 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:41.557504    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-7777c48764-pj5gk" podStartSLOduration=1.682704922 podStartE2EDuration="3.557488893s" totalImagesPullingTime="1.874783971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:41.557320845 +0000 UTC m=+502.622652822" watchObservedRunningTime="2026-07-06 23:56:41.557488893 +0000 UTC m=+502.622820871"
Jul 06 23:56:47.517683 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:47.517638    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-7777c48764-pj5gk"
Jul 06 23:56:52.514100 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:52.514063    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-7jv49"
Jul 06 23:56:52.514598 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:52.514154    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-lnvtg"
Jul 06 23:56:59.077836 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.077795    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"]
Jul 06 23:56:59.081317 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.081297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.086739 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.086707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fcvzt\""
Jul 06 23:56:59.086846 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.086769    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 23:56:59.086898 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.086777    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 23:56:59.086898 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.086707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 23:56:59.094259 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.094232    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"]
Jul 06 23:56:59.166269 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.166237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd895005-8e94-49b1-ad66-d0015e051db8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.166511 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.166301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.166511 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.166351    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmnc5\" (UniqueName: \"kubernetes.io/projected/cd895005-8e94-49b1-ad66-d0015e051db8-kube-api-access-mmnc5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.166511 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.166378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.166511 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.166412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.166511 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.166462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.267837 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.267792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.267982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.267860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmnc5\" (UniqueName: \"kubernetes.io/projected/cd895005-8e94-49b1-ad66-d0015e051db8-kube-api-access-mmnc5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.267982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.267896    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.267982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.267923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.267982 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.267949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.268201 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.268014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd895005-8e94-49b1-ad66-d0015e051db8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.268312 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.268288    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.268375 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.268311    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.268375 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.268357    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.270386 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.270364    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd895005-8e94-49b1-ad66-d0015e051db8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.270768 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.270750    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd895005-8e94-49b1-ad66-d0015e051db8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.277037 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.277008    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmnc5\" (UniqueName: \"kubernetes.io/projected/cd895005-8e94-49b1-ad66-d0015e051db8-kube-api-access-mmnc5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb\" (UID: \"cd895005-8e94-49b1-ad66-d0015e051db8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.391110 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.391025    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:56:59.532622 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.532587    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"]
Jul 06 23:56:59.536274 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:56:59.536238    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd895005_8e94_49b1_ad66_d0015e051db8.slice/crio-8383e565a0faf7543e05bc3dc1360ae30195fd6495a4fa9fb5432a83eac24d42 WatchSource:0}: Error finding container 8383e565a0faf7543e05bc3dc1360ae30195fd6495a4fa9fb5432a83eac24d42: Status 404 returned error can't find the container with id 8383e565a0faf7543e05bc3dc1360ae30195fd6495a4fa9fb5432a83eac24d42
Jul 06 23:56:59.585799 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:56:59.585765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb" event={"ID":"cd895005-8e94-49b1-ad66-d0015e051db8","Type":"ContainerStarted","Data":"8383e565a0faf7543e05bc3dc1360ae30195fd6495a4fa9fb5432a83eac24d42"}
Jul 06 23:57:01.602678 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:01.602642    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb" event={"ID":"cd895005-8e94-49b1-ad66-d0015e051db8","Type":"ContainerStarted","Data":"baa413b2e4a0c42a81ed2bcc19d517f5e78cfec0829b53c9658b5163dcbf6fa1"}
Jul 06 23:57:01.619558 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:01.619498    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb" podStartSLOduration=1.488084539 podStartE2EDuration="2.619481364s" totalImagesPullingTime="1.131396825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:01.61863417 +0000 UTC m=+522.683966147" watchObservedRunningTime="2026-07-06 23:57:01.619481364 +0000 UTC m=+522.684813391"
Jul 06 23:57:09.391705 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:09.391671    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:57:09.392194 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:09.391730    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:57:09.404494 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:09.404467    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:57:09.650923 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:09.650829    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb"
Jul 06 23:57:47.049186 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.049148    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"]
Jul 06 23:57:47.052908 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.052889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.056399 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.056373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 06 23:57:47.074507 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.074468    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"]
Jul 06 23:57:47.191842 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.191800    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.191842 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.191842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.192131 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.191893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76vzx\" (UniqueName: \"kubernetes.io/projected/875b2e5e-9377-40cf-af33-b38ee9777b6d-kube-api-access-76vzx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.192131 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.191921    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/875b2e5e-9377-40cf-af33-b38ee9777b6d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.192131 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.192061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.192131 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.192112    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.293380 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.293380 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.293607 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293505    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76vzx\" (UniqueName: \"kubernetes.io/projected/875b2e5e-9377-40cf-af33-b38ee9777b6d-kube-api-access-76vzx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.293607 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293549    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/875b2e5e-9377-40cf-af33-b38ee9777b6d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.293693 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.293693 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.293860 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293820    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.294015 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.293992    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.294078 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.294034    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.295820 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.295788    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/875b2e5e-9377-40cf-af33-b38ee9777b6d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.296124 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.296107    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/875b2e5e-9377-40cf-af33-b38ee9777b6d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.304726 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.304663    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76vzx\" (UniqueName: \"kubernetes.io/projected/875b2e5e-9377-40cf-af33-b38ee9777b6d-kube-api-access-76vzx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z\" (UID: \"875b2e5e-9377-40cf-af33-b38ee9777b6d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.362327 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.362288    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:47.510438 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.510396    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"]
Jul 06 23:57:47.513143 ip-10-0-130-92 kubenswrapper[2564]: W0706 23:57:47.513117    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod875b2e5e_9377_40cf_af33_b38ee9777b6d.slice/crio-d94afdbf673b0ae5067dec2f0b488b04e6569b4bf13d7c27d616402c7837f9f0 WatchSource:0}: Error finding container d94afdbf673b0ae5067dec2f0b488b04e6569b4bf13d7c27d616402c7837f9f0: Status 404 returned error can't find the container with id d94afdbf673b0ae5067dec2f0b488b04e6569b4bf13d7c27d616402c7837f9f0
Jul 06 23:57:47.799882 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.799840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z" event={"ID":"875b2e5e-9377-40cf-af33-b38ee9777b6d","Type":"ContainerStarted","Data":"4bc1b6ba01085a419843f13662a409e2ca3ba854abb9c989a2d4c489c9226ee9"}
Jul 06 23:57:47.799882 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.799885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z" event={"ID":"875b2e5e-9377-40cf-af33-b38ee9777b6d","Type":"ContainerStarted","Data":"d94afdbf673b0ae5067dec2f0b488b04e6569b4bf13d7c27d616402c7837f9f0"}
Jul 06 23:57:47.819966 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:47.819909    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z" podStartSLOduration=0.638958884 podStartE2EDuration="819.892076ms" totalImagesPullingTime="180.933192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:47.817556999 +0000 UTC m=+568.882889001" watchObservedRunningTime="2026-07-06 23:57:47.819892076 +0000 UTC m=+568.885224100"
Jul 06 23:57:57.362954 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:57.362918    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:57.362954 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:57.362966    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:57.375394 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:57.375367    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:57:57.857307 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:57:57.857278    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z"
Jul 06 23:58:19.292413 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:58:19.292378    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 06 23:58:19.292876 ip-10-0-130-92 kubenswrapper[2564]: I0706 23:58:19.292690    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:00:00.281990 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.281962    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67c88b8ddf-tf74m"]
Jul 07 00:00:00.285412 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.285390    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.290732 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.290710    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bhtjd\""
Jul 07 00:00:00.291162 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.291145    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 00:00:00.291944 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.291929    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 00:00:00.313567 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.313526    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c88b8ddf-tf74m"]
Jul 07 00:00:00.371509 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.371462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh244\" (UniqueName: \"kubernetes.io/projected/83df5ec1-4d14-4642-9d27-74e3ae5f1f92-kube-api-access-mh244\") pod \"maas-controller-67c88b8ddf-tf74m\" (UID: \"83df5ec1-4d14-4642-9d27-74e3ae5f1f92\") " pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.371676 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.371528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83df5ec1-4d14-4642-9d27-74e3ae5f1f92-webhook-cert\") pod \"maas-controller-67c88b8ddf-tf74m\" (UID: \"83df5ec1-4d14-4642-9d27-74e3ae5f1f92\") " pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.473023 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.472980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh244\" (UniqueName: \"kubernetes.io/projected/83df5ec1-4d14-4642-9d27-74e3ae5f1f92-kube-api-access-mh244\") pod \"maas-controller-67c88b8ddf-tf74m\" (UID: \"83df5ec1-4d14-4642-9d27-74e3ae5f1f92\") " pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.473198 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.473050    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83df5ec1-4d14-4642-9d27-74e3ae5f1f92-webhook-cert\") pod \"maas-controller-67c88b8ddf-tf74m\" (UID: \"83df5ec1-4d14-4642-9d27-74e3ae5f1f92\") " pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.475794 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.475765    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83df5ec1-4d14-4642-9d27-74e3ae5f1f92-webhook-cert\") pod \"maas-controller-67c88b8ddf-tf74m\" (UID: \"83df5ec1-4d14-4642-9d27-74e3ae5f1f92\") " pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.485381 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.485352    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh244\" (UniqueName: \"kubernetes.io/projected/83df5ec1-4d14-4642-9d27-74e3ae5f1f92-kube-api-access-mh244\") pod \"maas-controller-67c88b8ddf-tf74m\" (UID: \"83df5ec1-4d14-4642-9d27-74e3ae5f1f92\") " pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.611630 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.611546    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:00.757170 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.757145    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c88b8ddf-tf74m"]
Jul 07 00:00:00.759588 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:00:00.759561    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83df5ec1_4d14_4642_9d27_74e3ae5f1f92.slice/crio-3c3545394e12b68e541feaabe9f44bfa5f29dbd448098286da628c94e0b07ced WatchSource:0}: Error finding container 3c3545394e12b68e541feaabe9f44bfa5f29dbd448098286da628c94e0b07ced: Status 404 returned error can't find the container with id 3c3545394e12b68e541feaabe9f44bfa5f29dbd448098286da628c94e0b07ced
Jul 07 00:00:00.761127 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:00.761107    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:00:01.381039 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:01.381002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c88b8ddf-tf74m" event={"ID":"83df5ec1-4d14-4642-9d27-74e3ae5f1f92","Type":"ContainerStarted","Data":"3c3545394e12b68e541feaabe9f44bfa5f29dbd448098286da628c94e0b07ced"}
Jul 07 00:00:02.386931 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:02.386834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c88b8ddf-tf74m" event={"ID":"83df5ec1-4d14-4642-9d27-74e3ae5f1f92","Type":"ContainerStarted","Data":"dadb704b46d1019cdcb53123003932989e8ba1cfe41bb541f5de5c9a05b9d324"}
Jul 07 00:00:02.386931 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:02.386912    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:00:02.404888 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:02.404842    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67c88b8ddf-tf74m" podStartSLOduration=1.06528098 podStartE2EDuration="2.404826549s" totalImagesPullingTime="1.339545569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:02.403003271 +0000 UTC m=+703.468335248" watchObservedRunningTime="2026-07-07 00:00:02.404826549 +0000 UTC m=+703.470158525"
Jul 07 00:00:13.397933 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:00:13.397860    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67c88b8ddf-tf74m"
Jul 07 00:03:19.317151 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:03:19.317125    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:03:19.318517 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:03:19.318498    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:04:44.100801 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:44.100704    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"]
Jul 07 00:04:44.103273 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:44.100962    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q" podUID="a0c565b2-a749-4742-a5a7-bd89149f617b" containerName="manager" containerID="cri-o://2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d" gracePeriod=10
Jul 07 00:04:46.952003 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:46.951974    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 07 00:04:47.025151 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.025067    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ccst\" (UniqueName: \"kubernetes.io/projected/a0c565b2-a749-4742-a5a7-bd89149f617b-kube-api-access-6ccst\") pod \"a0c565b2-a749-4742-a5a7-bd89149f617b\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") "
Jul 07 00:04:47.025310 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.025171    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0c565b2-a749-4742-a5a7-bd89149f617b-extensions-socket-volume\") pod \"a0c565b2-a749-4742-a5a7-bd89149f617b\" (UID: \"a0c565b2-a749-4742-a5a7-bd89149f617b\") "
Jul 07 00:04:47.025545 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.025521    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c565b2-a749-4742-a5a7-bd89149f617b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a0c565b2-a749-4742-a5a7-bd89149f617b" (UID: "a0c565b2-a749-4742-a5a7-bd89149f617b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:04:47.027189 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.027171    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c565b2-a749-4742-a5a7-bd89149f617b-kube-api-access-6ccst" (OuterVolumeSpecName: "kube-api-access-6ccst") pod "a0c565b2-a749-4742-a5a7-bd89149f617b" (UID: "a0c565b2-a749-4742-a5a7-bd89149f617b"). InnerVolumeSpecName "kube-api-access-6ccst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:47.126733 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.126694    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0c565b2-a749-4742-a5a7-bd89149f617b-extensions-socket-volume\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:47.126733 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.126725    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ccst\" (UniqueName: \"kubernetes.io/projected/a0c565b2-a749-4742-a5a7-bd89149f617b-kube-api-access-6ccst\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:47.655479 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.655377    2564 generic.go:356] "Generic (PLEG): container finished" podID="a0c565b2-a749-4742-a5a7-bd89149f617b" containerID="2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d" exitCode=0
Jul 07 00:04:47.655479 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.655446    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"
Jul 07 00:04:47.655479 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.655463    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q" event={"ID":"a0c565b2-a749-4742-a5a7-bd89149f617b","Type":"ContainerDied","Data":"2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d"}
Jul 07 00:04:47.655707 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.655529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q" event={"ID":"a0c565b2-a749-4742-a5a7-bd89149f617b","Type":"ContainerDied","Data":"d0699df29336f010c51b95ad3c1a1ced757675fa3d434d6b3be610b226a4d246"}
Jul 07 00:04:47.655707 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.655559    2564 scope.go:117] "RemoveContainer" containerID="2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d"
Jul 07 00:04:47.664687 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.664666    2564 scope.go:117] "RemoveContainer" containerID="2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d"
Jul 07 00:04:47.664954 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:04:47.664933    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d\": container with ID starting with 2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d not found: ID does not exist" containerID="2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d"
Jul 07 00:04:47.665032 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.664968    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d"} err="failed to get container status \"2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d\": rpc error: code = NotFound desc = could not find container \"2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d\": container with ID starting with 2d4682bddfd6c41ba9085ff4c132c2deda37f471dfe53266b0234ddd9d8d771d not found: ID does not exist"
Jul 07 00:04:47.673868 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.673844    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"]
Jul 07 00:04:47.676938 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:47.676917    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d9j7q"]
Jul 07 00:04:49.392399 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:04:49.392363    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0c565b2-a749-4742-a5a7-bd89149f617b" path="/var/lib/kubelet/pods/a0c565b2-a749-4742-a5a7-bd89149f617b/volumes"
Jul 07 00:05:45.243224 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.243185    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"]
Jul 07 00:05:45.243664 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.243536    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c565b2-a749-4742-a5a7-bd89149f617b" containerName="manager"
Jul 07 00:05:45.243664 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.243546    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c565b2-a749-4742-a5a7-bd89149f617b" containerName="manager"
Jul 07 00:05:45.243664 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.243598    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c565b2-a749-4742-a5a7-bd89149f617b" containerName="manager"
Jul 07 00:05:45.246227 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.246211    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.248662 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.248639    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 00:05:45.248799 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.248783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mh67n\""
Jul 07 00:05:45.248844 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.248813    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 00:05:45.259941 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.259914    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"]
Jul 07 00:05:45.335623 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.335587    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7899a232-f64b-488e-aae5-794960999bb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4wtbm\" (UID: \"7899a232-f64b-488e-aae5-794960999bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.335777 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.335653    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmt9r\" (UniqueName: \"kubernetes.io/projected/7899a232-f64b-488e-aae5-794960999bb4-kube-api-access-tmt9r\") pod \"kuadrant-operator-controller-manager-86878c769f-4wtbm\" (UID: \"7899a232-f64b-488e-aae5-794960999bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.436855 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.436813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7899a232-f64b-488e-aae5-794960999bb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4wtbm\" (UID: \"7899a232-f64b-488e-aae5-794960999bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.437049 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.436901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmt9r\" (UniqueName: \"kubernetes.io/projected/7899a232-f64b-488e-aae5-794960999bb4-kube-api-access-tmt9r\") pod \"kuadrant-operator-controller-manager-86878c769f-4wtbm\" (UID: \"7899a232-f64b-488e-aae5-794960999bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.437329 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.437306    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7899a232-f64b-488e-aae5-794960999bb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4wtbm\" (UID: \"7899a232-f64b-488e-aae5-794960999bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.448890 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.448868    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmt9r\" (UniqueName: \"kubernetes.io/projected/7899a232-f64b-488e-aae5-794960999bb4-kube-api-access-tmt9r\") pod \"kuadrant-operator-controller-manager-86878c769f-4wtbm\" (UID: \"7899a232-f64b-488e-aae5-794960999bb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.556578 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.556491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.693653 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.693630    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"]
Jul 07 00:05:45.695980 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:05:45.695955    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7899a232_f64b_488e_aae5_794960999bb4.slice/crio-d8c49e0211baef6d1d9e7cfd3a06574baa4bf8d24bdad5b21e2f4312edf900b4 WatchSource:0}: Error finding container d8c49e0211baef6d1d9e7cfd3a06574baa4bf8d24bdad5b21e2f4312edf900b4: Status 404 returned error can't find the container with id d8c49e0211baef6d1d9e7cfd3a06574baa4bf8d24bdad5b21e2f4312edf900b4
Jul 07 00:05:45.899491 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.899369    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm" event={"ID":"7899a232-f64b-488e-aae5-794960999bb4","Type":"ContainerStarted","Data":"accc6bacc9234230d25f9bedf0f3e7fc66ccc0143ec01afc96dd451c5989fbfa"}
Jul 07 00:05:45.899660 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.899556    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:05:45.899660 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.899589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm" event={"ID":"7899a232-f64b-488e-aae5-794960999bb4","Type":"ContainerStarted","Data":"d8c49e0211baef6d1d9e7cfd3a06574baa4bf8d24bdad5b21e2f4312edf900b4"}
Jul 07 00:05:45.918773 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:45.918729    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm" podStartSLOduration=0.9187162 podStartE2EDuration="918.7162ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:45.917683582 +0000 UTC m=+1046.983015560" watchObservedRunningTime="2026-07-07 00:05:45.9187162 +0000 UTC m=+1046.984048176"
Jul 07 00:05:56.905985 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:05:56.905959    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4wtbm"
Jul 07 00:08:19.350735 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:08:19.350706    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:08:19.352519 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:08:19.352496    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:11:58.092170 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.092127    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"]
Jul 07 00:11:58.095919 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.095897    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.098376 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.098355    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-980e0fb5-serving-cert\""
Jul 07 00:11:58.106972 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.106951    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"]
Jul 07 00:11:58.206851 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.206816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pzmc\" (UniqueName: \"kubernetes.io/projected/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-kube-api-access-8pzmc\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.207021 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.206870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-openshift-service-ca\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.207021 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.206983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-maas-api-tls\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.307533 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.307503    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-openshift-service-ca\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.307666 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.307566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-maas-api-tls\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.307666 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.307618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pzmc\" (UniqueName: \"kubernetes.io/projected/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-kube-api-access-8pzmc\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.308120 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.308099    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-openshift-service-ca\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.310081 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.310058    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-maas-api-tls\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.315913 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.315884    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pzmc\" (UniqueName: \"kubernetes.io/projected/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-kube-api-access-8pzmc\") pod \"maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") " pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.407480 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.407402    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:58.542531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.542504    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"]
Jul 07 00:11:58.544612 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:11:58.544580    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44115e9f_b64b_4ee8_b5f7_a66deb77dff2.slice/crio-5c0cd9835f21a4e15bdefb96f3c50bef09aebcdcdf89c726bab86da9421fa765 WatchSource:0}: Error finding container 5c0cd9835f21a4e15bdefb96f3c50bef09aebcdcdf89c726bab86da9421fa765: Status 404 returned error can't find the container with id 5c0cd9835f21a4e15bdefb96f3c50bef09aebcdcdf89c726bab86da9421fa765
Jul 07 00:11:58.546459 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:58.546410    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:11:59.553217 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:59.553173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8" event={"ID":"44115e9f-b64b-4ee8-b5f7-a66deb77dff2","Type":"ContainerStarted","Data":"14eefa08c25b77a678fe9c4d1ec0c6470a6890155dfca74023feb4e3d084be26"}
Jul 07 00:11:59.553217 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:59.553221    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8" event={"ID":"44115e9f-b64b-4ee8-b5f7-a66deb77dff2","Type":"ContainerStarted","Data":"5c0cd9835f21a4e15bdefb96f3c50bef09aebcdcdf89c726bab86da9421fa765"}
Jul 07 00:11:59.553683 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:59.553272    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:11:59.572185 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:11:59.572132    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8" podStartSLOduration=1.115486 podStartE2EDuration="1.572117114s" totalImagesPullingTime="456.631114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:11:59.570343491 +0000 UTC m=+1420.635675469" watchObservedRunningTime="2026-07-07 00:11:59.572117114 +0000 UTC m=+1420.637449156"
Jul 07 00:12:03.420784 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.420750    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"]
Jul 07 00:12:03.421317 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.420991    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8" podUID="44115e9f-b64b-4ee8-b5f7-a66deb77dff2" containerName="maas-api" containerID="cri-o://14eefa08c25b77a678fe9c4d1ec0c6470a6890155dfca74023feb4e3d084be26" gracePeriod=30
Jul 07 00:12:03.575512 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.575479    2564 generic.go:356] "Generic (PLEG): container finished" podID="44115e9f-b64b-4ee8-b5f7-a66deb77dff2" containerID="14eefa08c25b77a678fe9c4d1ec0c6470a6890155dfca74023feb4e3d084be26" exitCode=0
Jul 07 00:12:03.575703 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.575555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8" event={"ID":"44115e9f-b64b-4ee8-b5f7-a66deb77dff2","Type":"ContainerDied","Data":"14eefa08c25b77a678fe9c4d1ec0c6470a6890155dfca74023feb4e3d084be26"}
Jul 07 00:12:03.663761 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.663731    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:12:03.751928 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.751898    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pzmc\" (UniqueName: \"kubernetes.io/projected/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-kube-api-access-8pzmc\") pod \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") "
Jul 07 00:12:03.752117 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.751949    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-openshift-service-ca\") pod \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") "
Jul 07 00:12:03.752117 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.751969    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-maas-api-tls\") pod \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\" (UID: \"44115e9f-b64b-4ee8-b5f7-a66deb77dff2\") "
Jul 07 00:12:03.752284 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.752259    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44115e9f-b64b-4ee8-b5f7-a66deb77dff2" (UID: "44115e9f-b64b-4ee8-b5f7-a66deb77dff2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:12:03.754123 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.754100    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-kube-api-access-8pzmc" (OuterVolumeSpecName: "kube-api-access-8pzmc") pod "44115e9f-b64b-4ee8-b5f7-a66deb77dff2" (UID: "44115e9f-b64b-4ee8-b5f7-a66deb77dff2"). InnerVolumeSpecName "kube-api-access-8pzmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:12:03.754210 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.754184    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44115e9f-b64b-4ee8-b5f7-a66deb77dff2" (UID: "44115e9f-b64b-4ee8-b5f7-a66deb77dff2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:12:03.853102 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.853061    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pzmc\" (UniqueName: \"kubernetes.io/projected/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-kube-api-access-8pzmc\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:03.853102 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.853094    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:03.853102 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:03.853108    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44115e9f-b64b-4ee8-b5f7-a66deb77dff2-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:04.580919 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:04.580890    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"
Jul 07 00:12:04.580919 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:04.580909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8" event={"ID":"44115e9f-b64b-4ee8-b5f7-a66deb77dff2","Type":"ContainerDied","Data":"5c0cd9835f21a4e15bdefb96f3c50bef09aebcdcdf89c726bab86da9421fa765"}
Jul 07 00:12:04.581502 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:04.580967    2564 scope.go:117] "RemoveContainer" containerID="14eefa08c25b77a678fe9c4d1ec0c6470a6890155dfca74023feb4e3d084be26"
Jul 07 00:12:04.602877 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:04.602848    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"]
Jul 07 00:12:04.604852 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:04.604826    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-980e0fb5-9cf74bd4b-pcnw8"]
Jul 07 00:12:05.391402 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:05.391362    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44115e9f-b64b-4ee8-b5f7-a66deb77dff2" path="/var/lib/kubelet/pods/44115e9f-b64b-4ee8-b5f7-a66deb77dff2/volumes"
Jul 07 00:12:10.477250 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.477167    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"]
Jul 07 00:12:10.477644 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.477605    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44115e9f-b64b-4ee8-b5f7-a66deb77dff2" containerName="maas-api"
Jul 07 00:12:10.477644 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.477618    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="44115e9f-b64b-4ee8-b5f7-a66deb77dff2" containerName="maas-api"
Jul 07 00:12:10.477733 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.477685    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="44115e9f-b64b-4ee8-b5f7-a66deb77dff2" containerName="maas-api"
Jul 07 00:12:10.492116 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.492084    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.495647 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.495605    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-4fbb0c76-serving-cert\""
Jul 07 00:12:10.496038 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.496014    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"]
Jul 07 00:12:10.500260 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.500234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d45a20-407a-4262-8f29-8dc9120cce5e-openshift-service-ca\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.500399 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.500273    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxsgz\" (UniqueName: \"kubernetes.io/projected/c5d45a20-407a-4262-8f29-8dc9120cce5e-kube-api-access-mxsgz\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.500399 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.500374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d45a20-407a-4262-8f29-8dc9120cce5e-maas-api-tls\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.601482 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.601446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d45a20-407a-4262-8f29-8dc9120cce5e-maas-api-tls\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.601780 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.601522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d45a20-407a-4262-8f29-8dc9120cce5e-openshift-service-ca\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.601780 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.601544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxsgz\" (UniqueName: \"kubernetes.io/projected/c5d45a20-407a-4262-8f29-8dc9120cce5e-kube-api-access-mxsgz\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.602351 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.602320    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d45a20-407a-4262-8f29-8dc9120cce5e-openshift-service-ca\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.604216 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.604176    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d45a20-407a-4262-8f29-8dc9120cce5e-maas-api-tls\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.611976 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.611950    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxsgz\" (UniqueName: \"kubernetes.io/projected/c5d45a20-407a-4262-8f29-8dc9120cce5e-kube-api-access-mxsgz\") pod \"maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") " pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.805652 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.805559    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:10.941365 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:10.941302    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"]
Jul 07 00:12:10.943807 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:12:10.943776    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d45a20_407a_4262_8f29_8dc9120cce5e.slice/crio-0bbd2089917a53913ce166f7d8fb2a1e2a9b8e1d86a511f0e70a8aa26ad85ccc WatchSource:0}: Error finding container 0bbd2089917a53913ce166f7d8fb2a1e2a9b8e1d86a511f0e70a8aa26ad85ccc: Status 404 returned error can't find the container with id 0bbd2089917a53913ce166f7d8fb2a1e2a9b8e1d86a511f0e70a8aa26ad85ccc
Jul 07 00:12:11.614637 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:11.614557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw" event={"ID":"c5d45a20-407a-4262-8f29-8dc9120cce5e","Type":"ContainerStarted","Data":"0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6"}
Jul 07 00:12:11.614637 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:11.614613    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw" event={"ID":"c5d45a20-407a-4262-8f29-8dc9120cce5e","Type":"ContainerStarted","Data":"0bbd2089917a53913ce166f7d8fb2a1e2a9b8e1d86a511f0e70a8aa26ad85ccc"}
Jul 07 00:12:11.615030 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:11.614646    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:11.658497 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:11.658412    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw" podStartSLOduration=1.275008315 podStartE2EDuration="1.658362384s" totalImagesPullingTime="383.354069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:12:11.657530432 +0000 UTC m=+1432.722862420" watchObservedRunningTime="2026-07-07 00:12:11.658362384 +0000 UTC m=+1432.723694363"
Jul 07 00:12:15.904146 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:15.904112    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"]
Jul 07 00:12:15.904676 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:15.904368    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw" podUID="c5d45a20-407a-4262-8f29-8dc9120cce5e" containerName="maas-api" containerID="cri-o://0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6" gracePeriod=30
Jul 07 00:12:16.156512 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.156443    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:16.239525 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.239494    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d45a20-407a-4262-8f29-8dc9120cce5e-openshift-service-ca\") pod \"c5d45a20-407a-4262-8f29-8dc9120cce5e\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") "
Jul 07 00:12:16.239525 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.239525    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d45a20-407a-4262-8f29-8dc9120cce5e-maas-api-tls\") pod \"c5d45a20-407a-4262-8f29-8dc9120cce5e\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") "
Jul 07 00:12:16.239730 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.239555    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxsgz\" (UniqueName: \"kubernetes.io/projected/c5d45a20-407a-4262-8f29-8dc9120cce5e-kube-api-access-mxsgz\") pod \"c5d45a20-407a-4262-8f29-8dc9120cce5e\" (UID: \"c5d45a20-407a-4262-8f29-8dc9120cce5e\") "
Jul 07 00:12:16.239926 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.239898    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d45a20-407a-4262-8f29-8dc9120cce5e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5d45a20-407a-4262-8f29-8dc9120cce5e" (UID: "c5d45a20-407a-4262-8f29-8dc9120cce5e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:12:16.241771 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.241743    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5d45a20-407a-4262-8f29-8dc9120cce5e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5d45a20-407a-4262-8f29-8dc9120cce5e" (UID: "c5d45a20-407a-4262-8f29-8dc9120cce5e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:12:16.241878 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.241797    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d45a20-407a-4262-8f29-8dc9120cce5e-kube-api-access-mxsgz" (OuterVolumeSpecName: "kube-api-access-mxsgz") pod "c5d45a20-407a-4262-8f29-8dc9120cce5e" (UID: "c5d45a20-407a-4262-8f29-8dc9120cce5e"). InnerVolumeSpecName "kube-api-access-mxsgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:12:16.340309 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.340275    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d45a20-407a-4262-8f29-8dc9120cce5e-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:16.340309 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.340304    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d45a20-407a-4262-8f29-8dc9120cce5e-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:16.340309 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.340313    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxsgz\" (UniqueName: \"kubernetes.io/projected/c5d45a20-407a-4262-8f29-8dc9120cce5e-kube-api-access-mxsgz\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:16.637492 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.637452    2564 generic.go:356] "Generic (PLEG): container finished" podID="c5d45a20-407a-4262-8f29-8dc9120cce5e" containerID="0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6" exitCode=0
Jul 07 00:12:16.637492 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.637490    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"
Jul 07 00:12:16.637697 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.637520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw" event={"ID":"c5d45a20-407a-4262-8f29-8dc9120cce5e","Type":"ContainerDied","Data":"0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6"}
Jul 07 00:12:16.637697 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.637574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw" event={"ID":"c5d45a20-407a-4262-8f29-8dc9120cce5e","Type":"ContainerDied","Data":"0bbd2089917a53913ce166f7d8fb2a1e2a9b8e1d86a511f0e70a8aa26ad85ccc"}
Jul 07 00:12:16.637697 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.637596    2564 scope.go:117] "RemoveContainer" containerID="0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6"
Jul 07 00:12:16.646732 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.646712    2564 scope.go:117] "RemoveContainer" containerID="0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6"
Jul 07 00:12:16.647000 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:12:16.646982    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6\": container with ID starting with 0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6 not found: ID does not exist" containerID="0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6"
Jul 07 00:12:16.647058 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.647007    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6"} err="failed to get container status \"0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6\": rpc error: code = NotFound desc = could not find container \"0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6\": container with ID starting with 0ac2b3fa9daec808a34891c955d126aaf17110cd867eee24b7064ef18ce2f1a6 not found: ID does not exist"
Jul 07 00:12:16.658319 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.658296    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"]
Jul 07 00:12:16.660733 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:16.660713    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-4fbb0c76-669fdbf9c5-mhppw"]
Jul 07 00:12:17.391352 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:17.391307    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5d45a20-407a-4262-8f29-8dc9120cce5e" path="/var/lib/kubelet/pods/c5d45a20-407a-4262-8f29-8dc9120cce5e/volumes"
Jul 07 00:12:29.235239 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.235203    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"]
Jul 07 00:12:29.235678 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.235588    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d45a20-407a-4262-8f29-8dc9120cce5e" containerName="maas-api"
Jul 07 00:12:29.235678 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.235602    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d45a20-407a-4262-8f29-8dc9120cce5e" containerName="maas-api"
Jul 07 00:12:29.235678 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.235674    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d45a20-407a-4262-8f29-8dc9120cce5e" containerName="maas-api"
Jul 07 00:12:29.240177 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.240156    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.244488 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.244462    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-5ddc79bd-serving-cert\""
Jul 07 00:12:29.257945 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.257916    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"]
Jul 07 00:12:29.353118 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.353083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b77efdf-8303-4730-8cb8-4bd1ef467e39-maas-api-tls\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.353313 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.353160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b77efdf-8303-4730-8cb8-4bd1ef467e39-openshift-service-ca\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.353313 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.353210    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46vmq\" (UniqueName: \"kubernetes.io/projected/2b77efdf-8303-4730-8cb8-4bd1ef467e39-kube-api-access-46vmq\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.454824 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.454787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b77efdf-8303-4730-8cb8-4bd1ef467e39-maas-api-tls\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.454824 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.454829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b77efdf-8303-4730-8cb8-4bd1ef467e39-openshift-service-ca\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.455075 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.454858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46vmq\" (UniqueName: \"kubernetes.io/projected/2b77efdf-8303-4730-8cb8-4bd1ef467e39-kube-api-access-46vmq\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.455620 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.455570    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b77efdf-8303-4730-8cb8-4bd1ef467e39-openshift-service-ca\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.457578 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.457552    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b77efdf-8303-4730-8cb8-4bd1ef467e39-maas-api-tls\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.463301 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.463274    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46vmq\" (UniqueName: \"kubernetes.io/projected/2b77efdf-8303-4730-8cb8-4bd1ef467e39-kube-api-access-46vmq\") pod \"maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") " pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.551361 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.551277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:29.696482 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.696457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"]
Jul 07 00:12:29.698727 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:12:29.698693    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b77efdf_8303_4730_8cb8_4bd1ef467e39.slice/crio-59b341c3ba164f89ad7cbbe4fa7995f3052f6e296ff1f9e24d981dcf213dd1bf WatchSource:0}: Error finding container 59b341c3ba164f89ad7cbbe4fa7995f3052f6e296ff1f9e24d981dcf213dd1bf: Status 404 returned error can't find the container with id 59b341c3ba164f89ad7cbbe4fa7995f3052f6e296ff1f9e24d981dcf213dd1bf
Jul 07 00:12:29.709716 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:29.709688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v" event={"ID":"2b77efdf-8303-4730-8cb8-4bd1ef467e39","Type":"ContainerStarted","Data":"59b341c3ba164f89ad7cbbe4fa7995f3052f6e296ff1f9e24d981dcf213dd1bf"}
Jul 07 00:12:30.715659 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:30.715624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v" event={"ID":"2b77efdf-8303-4730-8cb8-4bd1ef467e39","Type":"ContainerStarted","Data":"459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72"}
Jul 07 00:12:30.716044 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:30.715731    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:30.732734 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:30.732670    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v" podStartSLOduration=1.367115677 podStartE2EDuration="1.732653619s" totalImagesPullingTime="365.537942ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:12:30.73198478 +0000 UTC m=+1451.797316790" watchObservedRunningTime="2026-07-07 00:12:30.732653619 +0000 UTC m=+1451.797985615"
Jul 07 00:12:33.819173 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:33.819131    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"]
Jul 07 00:12:33.819653 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:33.819375    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v" podUID="2b77efdf-8303-4730-8cb8-4bd1ef467e39" containerName="maas-api" containerID="cri-o://459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72" gracePeriod=30
Jul 07 00:12:34.070745 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.070678    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:34.201772 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.201735    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b77efdf-8303-4730-8cb8-4bd1ef467e39-maas-api-tls\") pod \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") "
Jul 07 00:12:34.201772 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.201778    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b77efdf-8303-4730-8cb8-4bd1ef467e39-openshift-service-ca\") pod \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") "
Jul 07 00:12:34.202035 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.201830    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46vmq\" (UniqueName: \"kubernetes.io/projected/2b77efdf-8303-4730-8cb8-4bd1ef467e39-kube-api-access-46vmq\") pod \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\" (UID: \"2b77efdf-8303-4730-8cb8-4bd1ef467e39\") "
Jul 07 00:12:34.202264 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.202233    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b77efdf-8303-4730-8cb8-4bd1ef467e39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2b77efdf-8303-4730-8cb8-4bd1ef467e39" (UID: "2b77efdf-8303-4730-8cb8-4bd1ef467e39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:12:34.204059 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.204039    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b77efdf-8303-4730-8cb8-4bd1ef467e39-kube-api-access-46vmq" (OuterVolumeSpecName: "kube-api-access-46vmq") pod "2b77efdf-8303-4730-8cb8-4bd1ef467e39" (UID: "2b77efdf-8303-4730-8cb8-4bd1ef467e39"). InnerVolumeSpecName "kube-api-access-46vmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:12:34.204169 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.204096    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b77efdf-8303-4730-8cb8-4bd1ef467e39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2b77efdf-8303-4730-8cb8-4bd1ef467e39" (UID: "2b77efdf-8303-4730-8cb8-4bd1ef467e39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:12:34.302519 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.302478    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b77efdf-8303-4730-8cb8-4bd1ef467e39-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:34.302519 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.302512    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b77efdf-8303-4730-8cb8-4bd1ef467e39-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:34.302519 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.302522    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46vmq\" (UniqueName: \"kubernetes.io/projected/2b77efdf-8303-4730-8cb8-4bd1ef467e39-kube-api-access-46vmq\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:12:34.736717 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.736679    2564 generic.go:356] "Generic (PLEG): container finished" podID="2b77efdf-8303-4730-8cb8-4bd1ef467e39" containerID="459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72" exitCode=0
Jul 07 00:12:34.736917 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.736728    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"
Jul 07 00:12:34.736917 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.736762    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v" event={"ID":"2b77efdf-8303-4730-8cb8-4bd1ef467e39","Type":"ContainerDied","Data":"459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72"}
Jul 07 00:12:34.736917 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.736820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v" event={"ID":"2b77efdf-8303-4730-8cb8-4bd1ef467e39","Type":"ContainerDied","Data":"59b341c3ba164f89ad7cbbe4fa7995f3052f6e296ff1f9e24d981dcf213dd1bf"}
Jul 07 00:12:34.736917 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.736850    2564 scope.go:117] "RemoveContainer" containerID="459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72"
Jul 07 00:12:34.745960 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.745938    2564 scope.go:117] "RemoveContainer" containerID="459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72"
Jul 07 00:12:34.746253 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:12:34.746233    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72\": container with ID starting with 459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72 not found: ID does not exist" containerID="459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72"
Jul 07 00:12:34.746353 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.746258    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72"} err="failed to get container status \"459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72\": rpc error: code = NotFound desc = could not find container \"459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72\": container with ID starting with 459231a7c0ef23e198f45aca19df5b34fc81e948ea085da03e7664ec07eebf72 not found: ID does not exist"
Jul 07 00:12:34.760853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.760830    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"]
Jul 07 00:12:34.764354 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:34.764324    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-5ddc79bd-6cf7d56b8b-z227v"]
Jul 07 00:12:35.390797 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:12:35.390767    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b77efdf-8303-4730-8cb8-4bd1ef467e39" path="/var/lib/kubelet/pods/2b77efdf-8303-4730-8cb8-4bd1ef467e39/volumes"
Jul 07 00:13:19.375975 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:13:19.375943    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:13:19.378310 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:13:19.378290    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:14:18.998951 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:18.998901    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"]
Jul 07 00:14:18.999533 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:18.999235    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b77efdf-8303-4730-8cb8-4bd1ef467e39" containerName="maas-api"
Jul 07 00:14:18.999533 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:18.999246    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b77efdf-8303-4730-8cb8-4bd1ef467e39" containerName="maas-api"
Jul 07 00:14:18.999533 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:18.999323    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b77efdf-8303-4730-8cb8-4bd1ef467e39" containerName="maas-api"
Jul 07 00:14:19.002413 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.002388    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.005023 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.005000    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4de63527-serving-cert\""
Jul 07 00:14:19.014722 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.014699    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"]
Jul 07 00:14:19.132944 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.132900    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v948\" (UniqueName: \"kubernetes.io/projected/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-kube-api-access-8v948\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.133135 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.132960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-maas-api-tls\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.133135 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.132988    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-openshift-service-ca\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.233635 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.233596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-openshift-service-ca\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.233863 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.233722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v948\" (UniqueName: \"kubernetes.io/projected/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-kube-api-access-8v948\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.233863 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.233767    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-maas-api-tls\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.234453 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.234391    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-openshift-service-ca\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.236230 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.236203    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4de63527-serving-cert\""
Jul 07 00:14:19.242115 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.242090    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v948\" (UniqueName: \"kubernetes.io/projected/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-kube-api-access-8v948\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.247006 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.246979    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-maas-api-tls\") pod \"maas-api-e2e-mt-4de63527-8679686f4b-z2t89\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") " pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.315751 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.315645    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:19.460880 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:19.460489    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"]
Jul 07 00:14:20.211057 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:20.211013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89" event={"ID":"10fc79b0-5a7d-4815-9dd9-1fd525090c2a","Type":"ContainerStarted","Data":"23825539c878520a47c6189d09eeea5ee66a3b253aa619f9f98e249779771607"}
Jul 07 00:14:20.211519 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:20.211090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89" event={"ID":"10fc79b0-5a7d-4815-9dd9-1fd525090c2a","Type":"ContainerStarted","Data":"f13f7c6714775bfc24cedb0f09cab67d9233955eb217532447ec37fdc3f33c7d"}
Jul 07 00:14:20.211519 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:20.211135    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:20.232166 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:20.232114    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89" podStartSLOduration=1.714253139 podStartE2EDuration="2.232099281s" totalImagesPullingTime="517.846142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:14:20.229903136 +0000 UTC m=+1561.295235116" watchObservedRunningTime="2026-07-07 00:14:20.232099281 +0000 UTC m=+1561.297431275"
Jul 07 00:14:26.222059 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:26.222028    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:31.132512 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.132473    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"]
Jul 07 00:14:31.132979 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.132712    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89" podUID="10fc79b0-5a7d-4815-9dd9-1fd525090c2a" containerName="maas-api" containerID="cri-o://23825539c878520a47c6189d09eeea5ee66a3b253aa619f9f98e249779771607" gracePeriod=30
Jul 07 00:14:31.216951 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.216903    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89" podUID="10fc79b0-5a7d-4815-9dd9-1fd525090c2a" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.39:8443/health\": dial tcp 10.134.0.39:8443: connect: connection refused"
Jul 07 00:14:31.269555 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.269522    2564 generic.go:356] "Generic (PLEG): container finished" podID="10fc79b0-5a7d-4815-9dd9-1fd525090c2a" containerID="23825539c878520a47c6189d09eeea5ee66a3b253aa619f9f98e249779771607" exitCode=0
Jul 07 00:14:31.269734 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.269602    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89" event={"ID":"10fc79b0-5a7d-4815-9dd9-1fd525090c2a","Type":"ContainerDied","Data":"23825539c878520a47c6189d09eeea5ee66a3b253aa619f9f98e249779771607"}
Jul 07 00:14:31.378052 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.378028    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:31.550118 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.550082    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v948\" (UniqueName: \"kubernetes.io/projected/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-kube-api-access-8v948\") pod \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") "
Jul 07 00:14:31.550296 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.550125    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-openshift-service-ca\") pod \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") "
Jul 07 00:14:31.550296 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.550177    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-maas-api-tls\") pod \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\" (UID: \"10fc79b0-5a7d-4815-9dd9-1fd525090c2a\") "
Jul 07 00:14:31.550602 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.550570    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10fc79b0-5a7d-4815-9dd9-1fd525090c2a" (UID: "10fc79b0-5a7d-4815-9dd9-1fd525090c2a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:14:31.552385 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.552365    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10fc79b0-5a7d-4815-9dd9-1fd525090c2a" (UID: "10fc79b0-5a7d-4815-9dd9-1fd525090c2a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:14:31.552504 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.552381    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-kube-api-access-8v948" (OuterVolumeSpecName: "kube-api-access-8v948") pod "10fc79b0-5a7d-4815-9dd9-1fd525090c2a" (UID: "10fc79b0-5a7d-4815-9dd9-1fd525090c2a"). InnerVolumeSpecName "kube-api-access-8v948". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:14:31.650987 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.650951    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v948\" (UniqueName: \"kubernetes.io/projected/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-kube-api-access-8v948\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:14:31.650987 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.650985    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:14:31.651191 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:31.650999    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10fc79b0-5a7d-4815-9dd9-1fd525090c2a-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:14:32.275499 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:32.275470    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"
Jul 07 00:14:32.275925 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:32.275477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89" event={"ID":"10fc79b0-5a7d-4815-9dd9-1fd525090c2a","Type":"ContainerDied","Data":"f13f7c6714775bfc24cedb0f09cab67d9233955eb217532447ec37fdc3f33c7d"}
Jul 07 00:14:32.275925 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:32.275591    2564 scope.go:117] "RemoveContainer" containerID="23825539c878520a47c6189d09eeea5ee66a3b253aa619f9f98e249779771607"
Jul 07 00:14:32.297147 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:32.297119    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"]
Jul 07 00:14:32.301674 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:32.301653    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-4de63527-8679686f4b-z2t89"]
Jul 07 00:14:33.390693 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:33.390663    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10fc79b0-5a7d-4815-9dd9-1fd525090c2a" path="/var/lib/kubelet/pods/10fc79b0-5a7d-4815-9dd9-1fd525090c2a/volumes"
Jul 07 00:14:59.410367 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.410333    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"]
Jul 07 00:14:59.410852 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.410662    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10fc79b0-5a7d-4815-9dd9-1fd525090c2a" containerName="maas-api"
Jul 07 00:14:59.410852 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.410673    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="10fc79b0-5a7d-4815-9dd9-1fd525090c2a" containerName="maas-api"
Jul 07 00:14:59.410852 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.410737    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="10fc79b0-5a7d-4815-9dd9-1fd525090c2a" containerName="maas-api"
Jul 07 00:14:59.413507 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.413489    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.416596 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.416575    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-c75b7974-serving-cert\""
Jul 07 00:14:59.426019 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.425993    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"]
Jul 07 00:14:59.492956 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.492930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.493196 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.493168    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj56q\" (UniqueName: \"kubernetes.io/projected/81141009-45ad-4c19-96c4-68bc0d43a7f3-kube-api-access-fj56q\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.493320 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.493298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81141009-45ad-4c19-96c4-68bc0d43a7f3-openshift-service-ca\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.594458 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.594413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81141009-45ad-4c19-96c4-68bc0d43a7f3-openshift-service-ca\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.594597 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.594475    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.594597 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.594518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj56q\" (UniqueName: \"kubernetes.io/projected/81141009-45ad-4c19-96c4-68bc0d43a7f3-kube-api-access-fj56q\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.594597 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:14:59.594588    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-c75b7974-serving-cert: secret "maas-api-e2e-mt-c75b7974-serving-cert" not found
Jul 07 00:14:59.594704 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:14:59.594665    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls podName:81141009-45ad-4c19-96c4-68bc0d43a7f3 nodeName:}" failed. No retries permitted until 2026-07-07 00:15:00.094642473 +0000 UTC m=+1601.159974431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls") pod "maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk" (UID: "81141009-45ad-4c19-96c4-68bc0d43a7f3") : secret "maas-api-e2e-mt-c75b7974-serving-cert" not found
Jul 07 00:14:59.595016 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.595000    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81141009-45ad-4c19-96c4-68bc0d43a7f3-openshift-service-ca\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:14:59.604191 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:14:59.604170    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj56q\" (UniqueName: \"kubernetes.io/projected/81141009-45ad-4c19-96c4-68bc0d43a7f3-kube-api-access-fj56q\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:00.099347 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:00.099307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:00.101776 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:00.101753    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls\") pod \"maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") " pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:00.323031 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:00.322995    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:00.460979 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:00.460954    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"]
Jul 07 00:15:00.463340 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:15:00.463304    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81141009_45ad_4c19_96c4_68bc0d43a7f3.slice/crio-a3758a5f9625bd5246a1feff0c2affb17dffe00dc5a9a2d3e9c81090ac0fd334 WatchSource:0}: Error finding container a3758a5f9625bd5246a1feff0c2affb17dffe00dc5a9a2d3e9c81090ac0fd334: Status 404 returned error can't find the container with id a3758a5f9625bd5246a1feff0c2affb17dffe00dc5a9a2d3e9c81090ac0fd334
Jul 07 00:15:01.408840 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:01.408795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk" event={"ID":"81141009-45ad-4c19-96c4-68bc0d43a7f3","Type":"ContainerStarted","Data":"4c6250336593f84d14309f9fbbb148700bb391ab3fb4f81c1234becfb18f2720"}
Jul 07 00:15:01.409037 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:01.408860    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk" event={"ID":"81141009-45ad-4c19-96c4-68bc0d43a7f3","Type":"ContainerStarted","Data":"a3758a5f9625bd5246a1feff0c2affb17dffe00dc5a9a2d3e9c81090ac0fd334"}
Jul 07 00:15:01.411457 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:01.409937    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:01.428484 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:01.428405    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk" podStartSLOduration=2.040433351 podStartE2EDuration="2.428383907s" totalImagesPullingTime="387.950556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:01.427296691 +0000 UTC m=+1602.492628667" watchObservedRunningTime="2026-07-07 00:15:01.428383907 +0000 UTC m=+1602.493715883"
Jul 07 00:15:07.206572 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.206538    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"]
Jul 07 00:15:07.209849 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.209832    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.214238 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.214217    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c75b7974\"/\"default-dockercfg-bd6mn\""
Jul 07 00:15:07.215680 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.215644    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c75b7974\"/\"openshift-service-ca.crt\""
Jul 07 00:15:07.215803 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.215653    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c75b7974\"/\"kube-root-ca.crt\""
Jul 07 00:15:07.215911 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.215886    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c75b7974\"/\"e2e-lifecycle-model-c75b7974-kserve-self-signed-certs\""
Jul 07 00:15:07.220089 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.220069    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"]
Jul 07 00:15:07.362623 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.362582    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-home\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.362803 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.362647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tls-certs\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.362803 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.362677    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7lp9\" (UniqueName: \"kubernetes.io/projected/d1adeba4-d5ea-4cf9-973b-5faa7c463253-kube-api-access-r7lp9\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.362803 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.362710    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-dshm\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.362803 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.362786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-model-cache\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.362958 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.362819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tmp-dir\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464290 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-model-cache\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464470 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tmp-dir\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464470 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-home\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464470 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tls-certs\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464470 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464412    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7lp9\" (UniqueName: \"kubernetes.io/projected/d1adeba4-d5ea-4cf9-973b-5faa7c463253-kube-api-access-r7lp9\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464470 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-dshm\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464775 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464745    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-model-cache\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464863 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464791    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-home\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.464930 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.464905    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tmp-dir\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.466607 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.466585    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-dshm\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.467006 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.466987    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tls-certs\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.472962 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.472932    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7lp9\" (UniqueName: \"kubernetes.io/projected/d1adeba4-d5ea-4cf9-973b-5faa7c463253-kube-api-access-r7lp9\") pod \"e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") " pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.518713 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.518681    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:07.661168 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:07.661143    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"]
Jul 07 00:15:07.663843 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:15:07.663815    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1adeba4_d5ea_4cf9_973b_5faa7c463253.slice/crio-c44d01cb06f673a2fd70a4d430763736938d254c8a4321e2e8e8d7801c7d860c WatchSource:0}: Error finding container c44d01cb06f673a2fd70a4d430763736938d254c8a4321e2e8e8d7801c7d860c: Status 404 returned error can't find the container with id c44d01cb06f673a2fd70a4d430763736938d254c8a4321e2e8e8d7801c7d860c
Jul 07 00:15:08.425156 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:08.425120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:08.441966 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:08.441927    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq" event={"ID":"d1adeba4-d5ea-4cf9-973b-5faa7c463253","Type":"ContainerStarted","Data":"a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d"}
Jul 07 00:15:08.442141 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:08.441973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq" event={"ID":"d1adeba4-d5ea-4cf9-973b-5faa7c463253","Type":"ContainerStarted","Data":"c44d01cb06f673a2fd70a4d430763736938d254c8a4321e2e8e8d7801c7d860c"}
Jul 07 00:15:08.464242 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:08.464190    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq" podStartSLOduration=1.464174176 podStartE2EDuration="1.464174176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:08.462667783 +0000 UTC m=+1609.527999762" watchObservedRunningTime="2026-07-07 00:15:08.464174176 +0000 UTC m=+1609.529506152"
Jul 07 00:15:17.519658 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:17.519623    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:17.522133 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:17.519676    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:17.532541 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:17.532519    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:18.496850 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:18.496814    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:31.379170 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.379132    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"]
Jul 07 00:15:31.379659 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.379380    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk" podUID="81141009-45ad-4c19-96c4-68bc0d43a7f3" containerName="maas-api" containerID="cri-o://4c6250336593f84d14309f9fbbb148700bb391ab3fb4f81c1234becfb18f2720" gracePeriod=30
Jul 07 00:15:31.543475 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.543379    2564 generic.go:356] "Generic (PLEG): container finished" podID="81141009-45ad-4c19-96c4-68bc0d43a7f3" containerID="4c6250336593f84d14309f9fbbb148700bb391ab3fb4f81c1234becfb18f2720" exitCode=0
Jul 07 00:15:31.543475 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.543454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk" event={"ID":"81141009-45ad-4c19-96c4-68bc0d43a7f3","Type":"ContainerDied","Data":"4c6250336593f84d14309f9fbbb148700bb391ab3fb4f81c1234becfb18f2720"}
Jul 07 00:15:31.623202 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.623179    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:31.681546 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.681461    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls\") pod \"81141009-45ad-4c19-96c4-68bc0d43a7f3\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") "
Jul 07 00:15:31.681546 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.681540    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81141009-45ad-4c19-96c4-68bc0d43a7f3-openshift-service-ca\") pod \"81141009-45ad-4c19-96c4-68bc0d43a7f3\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") "
Jul 07 00:15:31.681767 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.681578    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj56q\" (UniqueName: \"kubernetes.io/projected/81141009-45ad-4c19-96c4-68bc0d43a7f3-kube-api-access-fj56q\") pod \"81141009-45ad-4c19-96c4-68bc0d43a7f3\" (UID: \"81141009-45ad-4c19-96c4-68bc0d43a7f3\") "
Jul 07 00:15:31.681875 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.681845    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81141009-45ad-4c19-96c4-68bc0d43a7f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81141009-45ad-4c19-96c4-68bc0d43a7f3" (UID: "81141009-45ad-4c19-96c4-68bc0d43a7f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:15:31.683718 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.683691    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81141009-45ad-4c19-96c4-68bc0d43a7f3-kube-api-access-fj56q" (OuterVolumeSpecName: "kube-api-access-fj56q") pod "81141009-45ad-4c19-96c4-68bc0d43a7f3" (UID: "81141009-45ad-4c19-96c4-68bc0d43a7f3"). InnerVolumeSpecName "kube-api-access-fj56q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:15:31.683860 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.683839    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81141009-45ad-4c19-96c4-68bc0d43a7f3" (UID: "81141009-45ad-4c19-96c4-68bc0d43a7f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:15:31.783180 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.783130    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81141009-45ad-4c19-96c4-68bc0d43a7f3-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:31.783180 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.783177    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81141009-45ad-4c19-96c4-68bc0d43a7f3-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:31.783180 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:31.783188    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj56q\" (UniqueName: \"kubernetes.io/projected/81141009-45ad-4c19-96c4-68bc0d43a7f3-kube-api-access-fj56q\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:32.549929 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:32.549903    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"
Jul 07 00:15:32.550338 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:32.549929    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk" event={"ID":"81141009-45ad-4c19-96c4-68bc0d43a7f3","Type":"ContainerDied","Data":"a3758a5f9625bd5246a1feff0c2affb17dffe00dc5a9a2d3e9c81090ac0fd334"}
Jul 07 00:15:32.550338 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:32.549990    2564 scope.go:117] "RemoveContainer" containerID="4c6250336593f84d14309f9fbbb148700bb391ab3fb4f81c1234becfb18f2720"
Jul 07 00:15:32.570972 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:32.570945    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"]
Jul 07 00:15:32.574137 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:32.574114    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-c75b7974-5495b85b9b-2tchk"]
Jul 07 00:15:33.395625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:33.392569    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81141009-45ad-4c19-96c4-68bc0d43a7f3" path="/var/lib/kubelet/pods/81141009-45ad-4c19-96c4-68bc0d43a7f3/volumes"
Jul 07 00:15:42.167872 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:42.167836    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"]
Jul 07 00:15:57.188848 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.188783    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq" podUID="d1adeba4-d5ea-4cf9-973b-5faa7c463253" containerName="main" containerID="cri-o://a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d" gracePeriod=45
Jul 07 00:15:57.437335 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.437307    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:57.627810 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.627772    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-model-cache\") pod \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") "
Jul 07 00:15:57.627997 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.627825    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7lp9\" (UniqueName: \"kubernetes.io/projected/d1adeba4-d5ea-4cf9-973b-5faa7c463253-kube-api-access-r7lp9\") pod \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") "
Jul 07 00:15:57.627997 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.627850    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-home\") pod \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") "
Jul 07 00:15:57.627997 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.627886    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tmp-dir\") pod \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") "
Jul 07 00:15:57.627997 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.627922    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tls-certs\") pod \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") "
Jul 07 00:15:57.627997 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.627995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-dshm\") pod \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\" (UID: \"d1adeba4-d5ea-4cf9-973b-5faa7c463253\") "
Jul 07 00:15:57.628239 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.628079    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1adeba4-d5ea-4cf9-973b-5faa7c463253" (UID: "d1adeba4-d5ea-4cf9-973b-5faa7c463253"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:15:57.628239 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.628141    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-home" (OuterVolumeSpecName: "home") pod "d1adeba4-d5ea-4cf9-973b-5faa7c463253" (UID: "d1adeba4-d5ea-4cf9-973b-5faa7c463253"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:15:57.628239 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.628182    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1adeba4-d5ea-4cf9-973b-5faa7c463253" (UID: "d1adeba4-d5ea-4cf9-973b-5faa7c463253"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:15:57.628509 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.628301    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:57.628509 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.628320    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:57.628509 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.628334    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:57.630351 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.630324    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-dshm" (OuterVolumeSpecName: "dshm") pod "d1adeba4-d5ea-4cf9-973b-5faa7c463253" (UID: "d1adeba4-d5ea-4cf9-973b-5faa7c463253"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:15:57.630489 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.630360    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1adeba4-d5ea-4cf9-973b-5faa7c463253-kube-api-access-r7lp9" (OuterVolumeSpecName: "kube-api-access-r7lp9") pod "d1adeba4-d5ea-4cf9-973b-5faa7c463253" (UID: "d1adeba4-d5ea-4cf9-973b-5faa7c463253"). InnerVolumeSpecName "kube-api-access-r7lp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:15:57.630489 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.630452    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1adeba4-d5ea-4cf9-973b-5faa7c463253" (UID: "d1adeba4-d5ea-4cf9-973b-5faa7c463253"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:15:57.663684 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.663650    2564 generic.go:356] "Generic (PLEG): container finished" podID="d1adeba4-d5ea-4cf9-973b-5faa7c463253" containerID="a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d" exitCode=0
Jul 07 00:15:57.663853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.663702    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"
Jul 07 00:15:57.663853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.663702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq" event={"ID":"d1adeba4-d5ea-4cf9-973b-5faa7c463253","Type":"ContainerDied","Data":"a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d"}
Jul 07 00:15:57.663853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.663749    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq" event={"ID":"d1adeba4-d5ea-4cf9-973b-5faa7c463253","Type":"ContainerDied","Data":"c44d01cb06f673a2fd70a4d430763736938d254c8a4321e2e8e8d7801c7d860c"}
Jul 07 00:15:57.663853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.663767    2564 scope.go:117] "RemoveContainer" containerID="a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d"
Jul 07 00:15:57.672925 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.672907    2564 scope.go:117] "RemoveContainer" containerID="a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d"
Jul 07 00:15:57.673210 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:15:57.673190    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d\": container with ID starting with a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d not found: ID does not exist" containerID="a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d"
Jul 07 00:15:57.673295 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.673216    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d"} err="failed to get container status \"a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d\": rpc error: code = NotFound desc = could not find container \"a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d\": container with ID starting with a53b32937561768662ae94bcdadc1c6beb475f75ed263ac7971fd512e93bf69d not found: ID does not exist"
Jul 07 00:15:57.684970 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.684938    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"]
Jul 07 00:15:57.687694 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.687668    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c75b7974/e2e-lifecycle-model-c75b7974-kserve-785b984699-fb7mq"]
Jul 07 00:15:57.728805 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.728775    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1adeba4-d5ea-4cf9-973b-5faa7c463253-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:57.728805 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.728804    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1adeba4-d5ea-4cf9-973b-5faa7c463253-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:57.728805 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:57.728813    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7lp9\" (UniqueName: \"kubernetes.io/projected/d1adeba4-d5ea-4cf9-973b-5faa7c463253-kube-api-access-r7lp9\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:15:59.396946 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:15:59.396910    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1adeba4-d5ea-4cf9-973b-5faa7c463253" path="/var/lib/kubelet/pods/d1adeba4-d5ea-4cf9-973b-5faa7c463253/volumes"
Jul 07 00:17:23.107868 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.107831    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"]
Jul 07 00:17:23.108346 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.108188    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1adeba4-d5ea-4cf9-973b-5faa7c463253" containerName="main"
Jul 07 00:17:23.108346 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.108202    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1adeba4-d5ea-4cf9-973b-5faa7c463253" containerName="main"
Jul 07 00:17:23.108346 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.108215    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81141009-45ad-4c19-96c4-68bc0d43a7f3" containerName="maas-api"
Jul 07 00:17:23.108346 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.108220    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="81141009-45ad-4c19-96c4-68bc0d43a7f3" containerName="maas-api"
Jul 07 00:17:23.108346 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.108295    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1adeba4-d5ea-4cf9-973b-5faa7c463253" containerName="main"
Jul 07 00:17:23.108346 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.108306    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="81141009-45ad-4c19-96c4-68bc0d43a7f3" containerName="maas-api"
Jul 07 00:17:23.111464 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.111445    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.114936 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.114911    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-30509d-serving-cert\""
Jul 07 00:17:23.124957 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.124932    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"]
Jul 07 00:17:23.274737 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.274690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.274737 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.274740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.274988 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.274855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqb9g\" (UniqueName: \"kubernetes.io/projected/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-kube-api-access-fqb9g\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.375979 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.375876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.375979 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.375923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.376235 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.376034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqb9g\" (UniqueName: \"kubernetes.io/projected/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-kube-api-access-fqb9g\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.376235 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:17:23.376047    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-30509d-serving-cert: secret "maas-api-e2e-inf-a-30509d-serving-cert" not found
Jul 07 00:17:23.376235 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:17:23.376150    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls podName:fe303e6e-0593-49d1-8ac7-7e4fcab345d3 nodeName:}" failed. No retries permitted until 2026-07-07 00:17:23.876127067 +0000 UTC m=+1744.941459022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls") pod "maas-api-e2e-inf-a-30509d-f9588787-rfm77" (UID: "fe303e6e-0593-49d1-8ac7-7e4fcab345d3") : secret "maas-api-e2e-inf-a-30509d-serving-cert" not found
Jul 07 00:17:23.376695 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.376673    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.384880 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.384855    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqb9g\" (UniqueName: \"kubernetes.io/projected/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-kube-api-access-fqb9g\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.879946 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.879907    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:23.882462 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:23.882436    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls\") pod \"maas-api-e2e-inf-a-30509d-f9588787-rfm77\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") " pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:24.022369 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:24.022326    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:24.181629 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:24.181600    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"]
Jul 07 00:17:24.184035 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:17:24.184010    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe303e6e_0593_49d1_8ac7_7e4fcab345d3.slice/crio-47efa10bbe3e24115762d0ab0f4728d8cbdf0f0f44f5516afd0ec574afdd0870 WatchSource:0}: Error finding container 47efa10bbe3e24115762d0ab0f4728d8cbdf0f0f44f5516afd0ec574afdd0870: Status 404 returned error can't find the container with id 47efa10bbe3e24115762d0ab0f4728d8cbdf0f0f44f5516afd0ec574afdd0870
Jul 07 00:17:24.185293 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:24.185273    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:17:25.052280 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:25.052245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77" event={"ID":"fe303e6e-0593-49d1-8ac7-7e4fcab345d3","Type":"ContainerStarted","Data":"27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869"}
Jul 07 00:17:25.052469 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:25.052298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77" event={"ID":"fe303e6e-0593-49d1-8ac7-7e4fcab345d3","Type":"ContainerStarted","Data":"47efa10bbe3e24115762d0ab0f4728d8cbdf0f0f44f5516afd0ec574afdd0870"}
Jul 07 00:17:25.052469 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:25.052354    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:25.070074 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:25.070021    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77" podStartSLOduration=1.555158372 podStartE2EDuration="2.070006351s" totalImagesPullingTime="514.847979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:25.067759265 +0000 UTC m=+1746.133091265" watchObservedRunningTime="2026-07-07 00:17:25.070006351 +0000 UTC m=+1746.135338328"
Jul 07 00:17:31.064757 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.064725    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:17:31.152125 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.152095    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"]
Jul 07 00:17:31.155742 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.155716    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.158478 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.158453    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-408dd6-serving-cert\""
Jul 07 00:17:31.167496 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.167236    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"]
Jul 07 00:17:31.250643 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.250610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f75005-6d97-4cb3-ab5d-634c5da60925-maas-api-tls\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.250822 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.250649    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f75005-6d97-4cb3-ab5d-634c5da60925-openshift-service-ca\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.250822 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.250667    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt88c\" (UniqueName: \"kubernetes.io/projected/a1f75005-6d97-4cb3-ab5d-634c5da60925-kube-api-access-qt88c\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.351490 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.351388    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f75005-6d97-4cb3-ab5d-634c5da60925-maas-api-tls\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.351490 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.351443    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f75005-6d97-4cb3-ab5d-634c5da60925-openshift-service-ca\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.351683 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.351573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt88c\" (UniqueName: \"kubernetes.io/projected/a1f75005-6d97-4cb3-ab5d-634c5da60925-kube-api-access-qt88c\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.352009 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.351991    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f75005-6d97-4cb3-ab5d-634c5da60925-openshift-service-ca\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.354094 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.354069    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f75005-6d97-4cb3-ab5d-634c5da60925-maas-api-tls\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.361204 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.361184    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt88c\" (UniqueName: \"kubernetes.io/projected/a1f75005-6d97-4cb3-ab5d-634c5da60925-kube-api-access-qt88c\") pod \"maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") " pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.468106 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.468053    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:31.626334 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:31.626306    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"]
Jul 07 00:17:31.629241 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:17:31.629213    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1f75005_6d97_4cb3_ab5d_634c5da60925.slice/crio-a719a660c1e1e864ab5948af352920c05a82671ce13eec0bb672496975174a5e WatchSource:0}: Error finding container a719a660c1e1e864ab5948af352920c05a82671ce13eec0bb672496975174a5e: Status 404 returned error can't find the container with id a719a660c1e1e864ab5948af352920c05a82671ce13eec0bb672496975174a5e
Jul 07 00:17:32.088595 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:32.088559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg" event={"ID":"a1f75005-6d97-4cb3-ab5d-634c5da60925","Type":"ContainerStarted","Data":"a719a660c1e1e864ab5948af352920c05a82671ce13eec0bb672496975174a5e"}
Jul 07 00:17:33.095249 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:33.095220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg" event={"ID":"a1f75005-6d97-4cb3-ab5d-634c5da60925","Type":"ContainerStarted","Data":"a2c7a7e5ca6bdfa5bcb971ae4d337adc52703f126eb71d6c283a945fc6845d21"}
Jul 07 00:17:33.095728 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:33.095351    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:17:33.112085 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:33.112037    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg" podStartSLOduration=1.699698579 podStartE2EDuration="2.112020194s" totalImagesPullingTime="412.321615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:33.111176873 +0000 UTC m=+1754.176508851" watchObservedRunningTime="2026-07-07 00:17:33.112020194 +0000 UTC m=+1754.177352171"
Jul 07 00:17:39.106194 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:17:39.106165    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:18:02.801773 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.801729    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"]
Jul 07 00:18:02.805830 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.805808    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:02.809715 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.809693    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-30509d\"/\"default-dockercfg-8f72j\""
Jul 07 00:18:02.809853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.809715    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-30509d\"/\"openshift-service-ca.crt\""
Jul 07 00:18:02.809853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.809794    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-30509d\"/\"kube-root-ca.crt\""
Jul 07 00:18:02.809853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.809815    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-30509d\"/\"test-model-30509d-kserve-self-signed-certs\""
Jul 07 00:18:02.813773 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.813750    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"]
Jul 07 00:18:02.915376 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.915337    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-dshm\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:02.915376 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.915377    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xtff\" (UniqueName: \"kubernetes.io/projected/ead83332-35bc-46f8-b391-3a26e05774d3-kube-api-access-6xtff\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:02.915630 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.915404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-model-cache\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:02.915630 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.915496    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead83332-35bc-46f8-b391-3a26e05774d3-tls-certs\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:02.915630 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.915535    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-home\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:02.915630 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:02.915566    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-tmp-dir\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.016612 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.016574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-dshm\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.016612 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.016614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xtff\" (UniqueName: \"kubernetes.io/projected/ead83332-35bc-46f8-b391-3a26e05774d3-kube-api-access-6xtff\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.016866 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.016637    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-model-cache\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.016866 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.016678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead83332-35bc-46f8-b391-3a26e05774d3-tls-certs\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.016866 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.016728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-home\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.016866 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.016771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-tmp-dir\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.017506 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.017475    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-model-cache\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.017688 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.017658    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-home\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.020067 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.020037    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead83332-35bc-46f8-b391-3a26e05774d3-tls-certs\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.023905 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.023881    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-tmp-dir\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.025094 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.024314    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-dshm\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.026131 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.026111    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xtff\" (UniqueName: \"kubernetes.io/projected/ead83332-35bc-46f8-b391-3a26e05774d3-kube-api-access-6xtff\") pod \"test-model-30509d-kserve-6d99c9f6c-2444d\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") " pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.116893 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.116809    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:03.249413 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:03.249352    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"]
Jul 07 00:18:03.252216 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:18:03.252182    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podead83332_35bc_46f8_b391_3a26e05774d3.slice/crio-9803e38dc120a98883e41a4c1ce142c57eff04b1d3fb040d4a28875f664990ef WatchSource:0}: Error finding container 9803e38dc120a98883e41a4c1ce142c57eff04b1d3fb040d4a28875f664990ef: Status 404 returned error can't find the container with id 9803e38dc120a98883e41a4c1ce142c57eff04b1d3fb040d4a28875f664990ef
Jul 07 00:18:04.233311 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:04.233269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d" event={"ID":"ead83332-35bc-46f8-b391-3a26e05774d3","Type":"ContainerStarted","Data":"cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7"}
Jul 07 00:18:04.233311 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:04.233318    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d" event={"ID":"ead83332-35bc-46f8-b391-3a26e05774d3","Type":"ContainerStarted","Data":"9803e38dc120a98883e41a4c1ce142c57eff04b1d3fb040d4a28875f664990ef"}
Jul 07 00:18:04.255718 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:04.255665    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d" podStartSLOduration=2.255649094 podStartE2EDuration="2.255649094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:18:04.254561159 +0000 UTC m=+1785.319893138" watchObservedRunningTime="2026-07-07 00:18:04.255649094 +0000 UTC m=+1785.320981127"
Jul 07 00:18:13.117814 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:13.117726    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:13.117814 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:13.117775    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:13.130277 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:13.130251    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:13.288436 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:13.288401    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:18:19.404451 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:19.404407    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:18:19.407118 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:19.407099    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:18:40.097993 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.097936    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"]
Jul 07 00:18:40.101652 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.101634    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.105903 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.105881    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-408dd6\"/\"kube-root-ca.crt\""
Jul 07 00:18:40.106017 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.105901    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-408dd6\"/\"test-model-408dd6-kserve-self-signed-certs\""
Jul 07 00:18:40.106017 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.105935    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-408dd6\"/\"openshift-service-ca.crt\""
Jul 07 00:18:40.106017 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.105882    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-408dd6\"/\"default-dockercfg-rcm8t\""
Jul 07 00:18:40.112732 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.112700    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"]
Jul 07 00:18:40.151984 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.151953    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-home\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.152125 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.152000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-tmp-dir\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.152125 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.152031    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113cfaf4-e612-45e0-b507-e20a497fadae-tls-certs\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.152125 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.152046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx8vp\" (UniqueName: \"kubernetes.io/projected/113cfaf4-e612-45e0-b507-e20a497fadae-kube-api-access-vx8vp\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.152276 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.152138    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-dshm\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.152276 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.152174    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-model-cache\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253435 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-home\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253600 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-tmp-dir\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253600 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113cfaf4-e612-45e0-b507-e20a497fadae-tls-certs\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253600 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253505    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8vp\" (UniqueName: \"kubernetes.io/projected/113cfaf4-e612-45e0-b507-e20a497fadae-kube-api-access-vx8vp\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253600 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-dshm\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253856 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253832    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-home\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253932 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253904    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-tmp-dir\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.253999 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.253909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-model-cache\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.254201 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.254174    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-model-cache\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.255860 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.255834    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-dshm\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.256080 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.256065    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113cfaf4-e612-45e0-b507-e20a497fadae-tls-certs\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.264615 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.264581    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx8vp\" (UniqueName: \"kubernetes.io/projected/113cfaf4-e612-45e0-b507-e20a497fadae-kube-api-access-vx8vp\") pod \"test-model-408dd6-kserve-574bd8c9b5-sntdv\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") " pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.411413 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.411332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:40.543848 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:40.543823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"]
Jul 07 00:18:40.546145 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:18:40.546105    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod113cfaf4_e612_45e0_b507_e20a497fadae.slice/crio-a64542972a12e7fcda29b141c40a0df39cf4094b0132c4ec242dd454d4240a2b WatchSource:0}: Error finding container a64542972a12e7fcda29b141c40a0df39cf4094b0132c4ec242dd454d4240a2b: Status 404 returned error can't find the container with id a64542972a12e7fcda29b141c40a0df39cf4094b0132c4ec242dd454d4240a2b
Jul 07 00:18:41.402101 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:41.402065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv" event={"ID":"113cfaf4-e612-45e0-b507-e20a497fadae","Type":"ContainerStarted","Data":"b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b"}
Jul 07 00:18:41.402101 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:41.402102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv" event={"ID":"113cfaf4-e612-45e0-b507-e20a497fadae","Type":"ContainerStarted","Data":"a64542972a12e7fcda29b141c40a0df39cf4094b0132c4ec242dd454d4240a2b"}
Jul 07 00:18:41.420365 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:41.420315    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv" podStartSLOduration=1.4203006710000001 podStartE2EDuration="1.420300671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:18:41.418853776 +0000 UTC m=+1822.484185752" watchObservedRunningTime="2026-07-07 00:18:41.420300671 +0000 UTC m=+1822.485632647"
Jul 07 00:18:50.412469 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:50.412412    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:50.412469 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:50.412475    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:50.425112 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:50.425082    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:50.455619 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:50.455593    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:18:59.733640 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:18:59.733605    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"]
Jul 07 00:19:00.635721 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:00.635688    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"]
Jul 07 00:19:01.529691 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:01.529635    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"]
Jul 07 00:19:01.530278 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:01.529883    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77" podUID="fe303e6e-0593-49d1-8ac7-7e4fcab345d3" containerName="maas-api" containerID="cri-o://27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869" gracePeriod=30
Jul 07 00:19:01.983363 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:01.983337    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:19:02.055296 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.055203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqb9g\" (UniqueName: \"kubernetes.io/projected/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-kube-api-access-fqb9g\") pod \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") "
Jul 07 00:19:02.055296 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.055282    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls\") pod \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") "
Jul 07 00:19:02.055531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.055301    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-openshift-service-ca\") pod \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\" (UID: \"fe303e6e-0593-49d1-8ac7-7e4fcab345d3\") "
Jul 07 00:19:02.055747 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.055724    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe303e6e-0593-49d1-8ac7-7e4fcab345d3" (UID: "fe303e6e-0593-49d1-8ac7-7e4fcab345d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:19:02.057560 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.057526    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe303e6e-0593-49d1-8ac7-7e4fcab345d3" (UID: "fe303e6e-0593-49d1-8ac7-7e4fcab345d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:19:02.057560 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.057550    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-kube-api-access-fqb9g" (OuterVolumeSpecName: "kube-api-access-fqb9g") pod "fe303e6e-0593-49d1-8ac7-7e4fcab345d3" (UID: "fe303e6e-0593-49d1-8ac7-7e4fcab345d3"). InnerVolumeSpecName "kube-api-access-fqb9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:19:02.156829 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.156793    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqb9g\" (UniqueName: \"kubernetes.io/projected/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-kube-api-access-fqb9g\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:02.156829 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.156825    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:02.156829 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.156835    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe303e6e-0593-49d1-8ac7-7e4fcab345d3-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:02.499798 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.499764    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe303e6e-0593-49d1-8ac7-7e4fcab345d3" containerID="27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869" exitCode=0
Jul 07 00:19:02.499986 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.499810    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"
Jul 07 00:19:02.499986 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.499819    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77" event={"ID":"fe303e6e-0593-49d1-8ac7-7e4fcab345d3","Type":"ContainerDied","Data":"27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869"}
Jul 07 00:19:02.499986 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.499861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77" event={"ID":"fe303e6e-0593-49d1-8ac7-7e4fcab345d3","Type":"ContainerDied","Data":"47efa10bbe3e24115762d0ab0f4728d8cbdf0f0f44f5516afd0ec574afdd0870"}
Jul 07 00:19:02.499986 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.499883    2564 scope.go:117] "RemoveContainer" containerID="27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869"
Jul 07 00:19:02.508863 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.508843    2564 scope.go:117] "RemoveContainer" containerID="27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869"
Jul 07 00:19:02.509169 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:19:02.509148    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869\": container with ID starting with 27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869 not found: ID does not exist" containerID="27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869"
Jul 07 00:19:02.509221 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.509177    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869"} err="failed to get container status \"27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869\": rpc error: code = NotFound desc = could not find container \"27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869\": container with ID starting with 27aae0b90057eed829871d5670fb8f8b0a0a53f1152145704fe5e2701c6c6869 not found: ID does not exist"
Jul 07 00:19:02.520288 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.520260    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"]
Jul 07 00:19:02.524326 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:02.524306    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-30509d-f9588787-rfm77"]
Jul 07 00:19:03.392002 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:03.391964    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe303e6e-0593-49d1-8ac7-7e4fcab345d3" path="/var/lib/kubelet/pods/fe303e6e-0593-49d1-8ac7-7e4fcab345d3/volumes"
Jul 07 00:19:14.754784 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:14.754741    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d" podUID="ead83332-35bc-46f8-b391-3a26e05774d3" containerName="main" containerID="cri-o://cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7" gracePeriod=45
Jul 07 00:19:15.010954 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.010878    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:19:15.170790 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.170754    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-tmp-dir\") pod \"ead83332-35bc-46f8-b391-3a26e05774d3\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") "
Jul 07 00:19:15.170790 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.170793    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-model-cache\") pod \"ead83332-35bc-46f8-b391-3a26e05774d3\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") "
Jul 07 00:19:15.171041 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.170817    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead83332-35bc-46f8-b391-3a26e05774d3-tls-certs\") pod \"ead83332-35bc-46f8-b391-3a26e05774d3\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") "
Jul 07 00:19:15.171041 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.170847    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-dshm\") pod \"ead83332-35bc-46f8-b391-3a26e05774d3\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") "
Jul 07 00:19:15.171041 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.170875    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xtff\" (UniqueName: \"kubernetes.io/projected/ead83332-35bc-46f8-b391-3a26e05774d3-kube-api-access-6xtff\") pod \"ead83332-35bc-46f8-b391-3a26e05774d3\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") "
Jul 07 00:19:15.171041 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.170953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-home\") pod \"ead83332-35bc-46f8-b391-3a26e05774d3\" (UID: \"ead83332-35bc-46f8-b391-3a26e05774d3\") "
Jul 07 00:19:15.171229 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.171082    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ead83332-35bc-46f8-b391-3a26e05774d3" (UID: "ead83332-35bc-46f8-b391-3a26e05774d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:15.171229 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.171091    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "ead83332-35bc-46f8-b391-3a26e05774d3" (UID: "ead83332-35bc-46f8-b391-3a26e05774d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:15.171342 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.171287    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-home" (OuterVolumeSpecName: "home") pod "ead83332-35bc-46f8-b391-3a26e05774d3" (UID: "ead83332-35bc-46f8-b391-3a26e05774d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:15.171394 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.171382    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:15.171486 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.171405    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:15.173295 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.173259    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-dshm" (OuterVolumeSpecName: "dshm") pod "ead83332-35bc-46f8-b391-3a26e05774d3" (UID: "ead83332-35bc-46f8-b391-3a26e05774d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:15.173395 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.173276    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead83332-35bc-46f8-b391-3a26e05774d3-kube-api-access-6xtff" (OuterVolumeSpecName: "kube-api-access-6xtff") pod "ead83332-35bc-46f8-b391-3a26e05774d3" (UID: "ead83332-35bc-46f8-b391-3a26e05774d3"). InnerVolumeSpecName "kube-api-access-6xtff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:19:15.173395 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.173370    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead83332-35bc-46f8-b391-3a26e05774d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ead83332-35bc-46f8-b391-3a26e05774d3" (UID: "ead83332-35bc-46f8-b391-3a26e05774d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:19:15.272923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.272839    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xtff\" (UniqueName: \"kubernetes.io/projected/ead83332-35bc-46f8-b391-3a26e05774d3-kube-api-access-6xtff\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:15.272923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.272868    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:15.272923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.272877    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ead83332-35bc-46f8-b391-3a26e05774d3-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:15.272923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.272885    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ead83332-35bc-46f8-b391-3a26e05774d3-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:15.565531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.565436    2564 generic.go:356] "Generic (PLEG): container finished" podID="ead83332-35bc-46f8-b391-3a26e05774d3" containerID="cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7" exitCode=0
Jul 07 00:19:15.565531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.565486    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"
Jul 07 00:19:15.565736 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.565524    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d" event={"ID":"ead83332-35bc-46f8-b391-3a26e05774d3","Type":"ContainerDied","Data":"cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7"}
Jul 07 00:19:15.565736 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.565570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d" event={"ID":"ead83332-35bc-46f8-b391-3a26e05774d3","Type":"ContainerDied","Data":"9803e38dc120a98883e41a4c1ce142c57eff04b1d3fb040d4a28875f664990ef"}
Jul 07 00:19:15.565736 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.565591    2564 scope.go:117] "RemoveContainer" containerID="cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7"
Jul 07 00:19:15.573883 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.573863    2564 scope.go:117] "RemoveContainer" containerID="cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7"
Jul 07 00:19:15.574137 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:19:15.574118    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7\": container with ID starting with cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7 not found: ID does not exist" containerID="cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7"
Jul 07 00:19:15.574203 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.574146    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7"} err="failed to get container status \"cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7\": rpc error: code = NotFound desc = could not find container \"cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7\": container with ID starting with cd5a16a887263bfacd6fb363d1bd684dfb4ca165fcd38f4f335f1ba60f133fa7 not found: ID does not exist"
Jul 07 00:19:15.581805 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.581783    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"]
Jul 07 00:19:15.586171 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.586139    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-30509d/test-model-30509d-kserve-6d99c9f6c-2444d"]
Jul 07 00:19:15.656713 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.656655    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv" podUID="113cfaf4-e612-45e0-b507-e20a497fadae" containerName="main" containerID="cri-o://b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b" gracePeriod=45
Jul 07 00:19:15.910799 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:15.910777    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:19:16.078505 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-home\") pod \"113cfaf4-e612-45e0-b507-e20a497fadae\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") "
Jul 07 00:19:16.078649 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078535    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-dshm\") pod \"113cfaf4-e612-45e0-b507-e20a497fadae\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") "
Jul 07 00:19:16.078649 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-model-cache\") pod \"113cfaf4-e612-45e0-b507-e20a497fadae\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") "
Jul 07 00:19:16.078649 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078591    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113cfaf4-e612-45e0-b507-e20a497fadae-tls-certs\") pod \"113cfaf4-e612-45e0-b507-e20a497fadae\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") "
Jul 07 00:19:16.078649 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078632    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx8vp\" (UniqueName: \"kubernetes.io/projected/113cfaf4-e612-45e0-b507-e20a497fadae-kube-api-access-vx8vp\") pod \"113cfaf4-e612-45e0-b507-e20a497fadae\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") "
Jul 07 00:19:16.078810 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078664    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-tmp-dir\") pod \"113cfaf4-e612-45e0-b507-e20a497fadae\" (UID: \"113cfaf4-e612-45e0-b507-e20a497fadae\") "
Jul 07 00:19:16.078810 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078716    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-home" (OuterVolumeSpecName: "home") pod "113cfaf4-e612-45e0-b507-e20a497fadae" (UID: "113cfaf4-e612-45e0-b507-e20a497fadae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:16.078927 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078826    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-model-cache" (OuterVolumeSpecName: "model-cache") pod "113cfaf4-e612-45e0-b507-e20a497fadae" (UID: "113cfaf4-e612-45e0-b507-e20a497fadae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:16.078987 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078953    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:16.078987 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.078967    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:16.079087 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.079023    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "113cfaf4-e612-45e0-b507-e20a497fadae" (UID: "113cfaf4-e612-45e0-b507-e20a497fadae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:16.080857 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.080833    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-dshm" (OuterVolumeSpecName: "dshm") pod "113cfaf4-e612-45e0-b507-e20a497fadae" (UID: "113cfaf4-e612-45e0-b507-e20a497fadae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:19:16.080959 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.080902    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113cfaf4-e612-45e0-b507-e20a497fadae-kube-api-access-vx8vp" (OuterVolumeSpecName: "kube-api-access-vx8vp") pod "113cfaf4-e612-45e0-b507-e20a497fadae" (UID: "113cfaf4-e612-45e0-b507-e20a497fadae"). InnerVolumeSpecName "kube-api-access-vx8vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:19:16.080996 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.080961    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113cfaf4-e612-45e0-b507-e20a497fadae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "113cfaf4-e612-45e0-b507-e20a497fadae" (UID: "113cfaf4-e612-45e0-b507-e20a497fadae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:19:16.179575 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.179550    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:16.179575 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.179573    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113cfaf4-e612-45e0-b507-e20a497fadae-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:16.179711 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.179585    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx8vp\" (UniqueName: \"kubernetes.io/projected/113cfaf4-e612-45e0-b507-e20a497fadae-kube-api-access-vx8vp\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:16.179711 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.179613    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113cfaf4-e612-45e0-b507-e20a497fadae-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:16.572676 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.572638    2564 generic.go:356] "Generic (PLEG): container finished" podID="113cfaf4-e612-45e0-b507-e20a497fadae" containerID="b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b" exitCode=0
Jul 07 00:19:16.572834 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.572683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv" event={"ID":"113cfaf4-e612-45e0-b507-e20a497fadae","Type":"ContainerDied","Data":"b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b"}
Jul 07 00:19:16.572834 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.572697    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"
Jul 07 00:19:16.572834 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.572719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv" event={"ID":"113cfaf4-e612-45e0-b507-e20a497fadae","Type":"ContainerDied","Data":"a64542972a12e7fcda29b141c40a0df39cf4094b0132c4ec242dd454d4240a2b"}
Jul 07 00:19:16.572834 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.572740    2564 scope.go:117] "RemoveContainer" containerID="b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b"
Jul 07 00:19:16.584697 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.584680    2564 scope.go:117] "RemoveContainer" containerID="b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b"
Jul 07 00:19:16.585182 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:19:16.585155    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b\": container with ID starting with b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b not found: ID does not exist" containerID="b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b"
Jul 07 00:19:16.585182 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.585193    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b"} err="failed to get container status \"b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b\": rpc error: code = NotFound desc = could not find container \"b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b\": container with ID starting with b494f3686a8bd0658e1fc7f23b86677303a23ff16989376f1e914db60e855b7b not found: ID does not exist"
Jul 07 00:19:16.595455 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.595417    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"]
Jul 07 00:19:16.600702 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:16.600682    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-408dd6/test-model-408dd6-kserve-574bd8c9b5-sntdv"]
Jul 07 00:19:17.390620 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:17.390587    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="113cfaf4-e612-45e0-b507-e20a497fadae" path="/var/lib/kubelet/pods/113cfaf4-e612-45e0-b507-e20a497fadae/volumes"
Jul 07 00:19:17.391012 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:17.390983    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ead83332-35bc-46f8-b391-3a26e05774d3" path="/var/lib/kubelet/pods/ead83332-35bc-46f8-b391-3a26e05774d3/volumes"
Jul 07 00:19:44.374554 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.374457    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"]
Jul 07 00:19:44.375152 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.374710    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg" podUID="a1f75005-6d97-4cb3-ab5d-634c5da60925" containerName="maas-api" containerID="cri-o://a2c7a7e5ca6bdfa5bcb971ae4d337adc52703f126eb71d6c283a945fc6845d21" gracePeriod=30
Jul 07 00:19:44.691756 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.691724    2564 generic.go:356] "Generic (PLEG): container finished" podID="a1f75005-6d97-4cb3-ab5d-634c5da60925" containerID="a2c7a7e5ca6bdfa5bcb971ae4d337adc52703f126eb71d6c283a945fc6845d21" exitCode=0
Jul 07 00:19:44.691868 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.691777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg" event={"ID":"a1f75005-6d97-4cb3-ab5d-634c5da60925","Type":"ContainerDied","Data":"a2c7a7e5ca6bdfa5bcb971ae4d337adc52703f126eb71d6c283a945fc6845d21"}
Jul 07 00:19:44.816757 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.816731    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:19:44.927464 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.927362    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f75005-6d97-4cb3-ab5d-634c5da60925-maas-api-tls\") pod \"a1f75005-6d97-4cb3-ab5d-634c5da60925\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") "
Jul 07 00:19:44.927620 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.927491    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt88c\" (UniqueName: \"kubernetes.io/projected/a1f75005-6d97-4cb3-ab5d-634c5da60925-kube-api-access-qt88c\") pod \"a1f75005-6d97-4cb3-ab5d-634c5da60925\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") "
Jul 07 00:19:44.927620 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.927561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f75005-6d97-4cb3-ab5d-634c5da60925-openshift-service-ca\") pod \"a1f75005-6d97-4cb3-ab5d-634c5da60925\" (UID: \"a1f75005-6d97-4cb3-ab5d-634c5da60925\") "
Jul 07 00:19:44.927925 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.927895    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1f75005-6d97-4cb3-ab5d-634c5da60925-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1f75005-6d97-4cb3-ab5d-634c5da60925" (UID: "a1f75005-6d97-4cb3-ab5d-634c5da60925"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:19:44.929606 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.929578    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1f75005-6d97-4cb3-ab5d-634c5da60925-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1f75005-6d97-4cb3-ab5d-634c5da60925" (UID: "a1f75005-6d97-4cb3-ab5d-634c5da60925"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:19:44.929693 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:44.929654    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1f75005-6d97-4cb3-ab5d-634c5da60925-kube-api-access-qt88c" (OuterVolumeSpecName: "kube-api-access-qt88c") pod "a1f75005-6d97-4cb3-ab5d-634c5da60925" (UID: "a1f75005-6d97-4cb3-ab5d-634c5da60925"). InnerVolumeSpecName "kube-api-access-qt88c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:19:45.028858 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.028830    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1f75005-6d97-4cb3-ab5d-634c5da60925-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:45.028858 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.028857    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1f75005-6d97-4cb3-ab5d-634c5da60925-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:45.029022 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.028868    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt88c\" (UniqueName: \"kubernetes.io/projected/a1f75005-6d97-4cb3-ab5d-634c5da60925-kube-api-access-qt88c\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:19:45.697109 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.697019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg" event={"ID":"a1f75005-6d97-4cb3-ab5d-634c5da60925","Type":"ContainerDied","Data":"a719a660c1e1e864ab5948af352920c05a82671ce13eec0bb672496975174a5e"}
Jul 07 00:19:45.697109 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.697083    2564 scope.go:117] "RemoveContainer" containerID="a2c7a7e5ca6bdfa5bcb971ae4d337adc52703f126eb71d6c283a945fc6845d21"
Jul 07 00:19:45.697109 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.697083    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"
Jul 07 00:19:45.713763 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.713731    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"]
Jul 07 00:19:45.717953 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:45.717928    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-408dd6-5f98b699bf-9rnsg"]
Jul 07 00:19:47.392897 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:47.392853    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1f75005-6d97-4cb3-ab5d-634c5da60925" path="/var/lib/kubelet/pods/a1f75005-6d97-4cb3-ab5d-634c5da60925/volumes"
Jul 07 00:19:54.125323 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125290    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"]
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125666    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113cfaf4-e612-45e0-b507-e20a497fadae" containerName="main"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125681    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="113cfaf4-e612-45e0-b507-e20a497fadae" containerName="main"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125693    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe303e6e-0593-49d1-8ac7-7e4fcab345d3" containerName="maas-api"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125699    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe303e6e-0593-49d1-8ac7-7e4fcab345d3" containerName="maas-api"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125721    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ead83332-35bc-46f8-b391-3a26e05774d3" containerName="main"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125727    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ead83332-35bc-46f8-b391-3a26e05774d3" containerName="main"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125738    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1f75005-6d97-4cb3-ab5d-634c5da60925" containerName="maas-api"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125747    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1f75005-6d97-4cb3-ab5d-634c5da60925" containerName="maas-api"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125801    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe303e6e-0593-49d1-8ac7-7e4fcab345d3" containerName="maas-api"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125809    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ead83332-35bc-46f8-b391-3a26e05774d3" containerName="main"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125816    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="113cfaf4-e612-45e0-b507-e20a497fadae" containerName="main"
Jul 07 00:19:54.125827 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.125828    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1f75005-6d97-4cb3-ab5d-634c5da60925" containerName="maas-api"
Jul 07 00:19:54.128811 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.128787    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.131678 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.131649    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-4d772e-serving-cert\""
Jul 07 00:19:54.142303 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.142273    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"]
Jul 07 00:19:54.205430 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.205396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-openshift-service-ca\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.205613 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.205474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.205613 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.205523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctllb\" (UniqueName: \"kubernetes.io/projected/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-kube-api-access-ctllb\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.306798 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.306756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.307010 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.306821    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctllb\" (UniqueName: \"kubernetes.io/projected/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-kube-api-access-ctllb\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.307010 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.306898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-openshift-service-ca\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.307010 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:19:54.306931    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-4d772e-serving-cert: secret "maas-api-e2e-api-a-4d772e-serving-cert" not found
Jul 07 00:19:54.307187 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:19:54.307018    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls podName:fa13fa83-6304-48ba-b3ef-0abfe95f1a37 nodeName:}" failed. No retries permitted until 2026-07-07 00:19:54.806996631 +0000 UTC m=+1895.872328590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls") pod "maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh" (UID: "fa13fa83-6304-48ba-b3ef-0abfe95f1a37") : secret "maas-api-e2e-api-a-4d772e-serving-cert" not found
Jul 07 00:19:54.307588 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.307566    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-openshift-service-ca\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.318985 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.318953    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctllb\" (UniqueName: \"kubernetes.io/projected/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-kube-api-access-ctllb\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.811317 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.811263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:54.813826 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:54.813803    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls\") pod \"maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") " pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:55.041084 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:55.041051    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:55.181326 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:55.181190    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"]
Jul 07 00:19:55.184129 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:19:55.184099    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa13fa83_6304_48ba_b3ef_0abfe95f1a37.slice/crio-a6ce76fbb20695ccdc4881faec520ea8693b8f63157fdaa26b4dee9d4546a0d1 WatchSource:0}: Error finding container a6ce76fbb20695ccdc4881faec520ea8693b8f63157fdaa26b4dee9d4546a0d1: Status 404 returned error can't find the container with id a6ce76fbb20695ccdc4881faec520ea8693b8f63157fdaa26b4dee9d4546a0d1
Jul 07 00:19:55.747190 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:55.747157    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh" event={"ID":"fa13fa83-6304-48ba-b3ef-0abfe95f1a37","Type":"ContainerStarted","Data":"f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e"}
Jul 07 00:19:55.747363 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:55.747210    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh" event={"ID":"fa13fa83-6304-48ba-b3ef-0abfe95f1a37","Type":"ContainerStarted","Data":"a6ce76fbb20695ccdc4881faec520ea8693b8f63157fdaa26b4dee9d4546a0d1"}
Jul 07 00:19:55.747363 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:55.747306    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:19:55.764238 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:19:55.764187    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh" podStartSLOduration=1.37008402 podStartE2EDuration="1.764172038s" totalImagesPullingTime="394.088018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:19:55.762780233 +0000 UTC m=+1896.828112211" watchObservedRunningTime="2026-07-07 00:19:55.764172038 +0000 UTC m=+1896.829504015"
Jul 07 00:20:01.756673 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:01.756642    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:20:02.122008 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.121916    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"]
Jul 07 00:20:02.125684 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.125663    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.128231 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.128204    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-5b1463-serving-cert\""
Jul 07 00:20:02.138103 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.138078    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"]
Jul 07 00:20:02.279409 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.279369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45zs4\" (UniqueName: \"kubernetes.io/projected/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-kube-api-access-45zs4\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.279409 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.279415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.279691 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.279541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-openshift-service-ca\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.380908 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.380814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-openshift-service-ca\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.380908 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.380875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45zs4\" (UniqueName: \"kubernetes.io/projected/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-kube-api-access-45zs4\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.380908 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.380899    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.381203 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:02.381015    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-5b1463-serving-cert: secret "maas-api-e2e-api-b-5b1463-serving-cert" not found
Jul 07 00:20:02.381203 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:02.381084    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls podName:16fa43c3-fd13-47bb-8d72-e4d02ddd15ae nodeName:}" failed. No retries permitted until 2026-07-07 00:20:02.881065567 +0000 UTC m=+1903.946397528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls") pod "maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn" (UID: "16fa43c3-fd13-47bb-8d72-e4d02ddd15ae") : secret "maas-api-e2e-api-b-5b1463-serving-cert" not found
Jul 07 00:20:02.381614 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.381588    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-openshift-service-ca\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.392464 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.392435    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45zs4\" (UniqueName: \"kubernetes.io/projected/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-kube-api-access-45zs4\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.887123 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.887090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:02.889682 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:02.889662    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls\") pod \"maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") " pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:03.036570 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:03.036535    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:03.172362 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:03.172327    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"]
Jul 07 00:20:03.173724 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:20:03.173699    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fa43c3_fd13_47bb_8d72_e4d02ddd15ae.slice/crio-a42e3ec699b3785aaa8c1139d3bacc1c6afa531b34f44511f2cfb77e3c670f0d WatchSource:0}: Error finding container a42e3ec699b3785aaa8c1139d3bacc1c6afa531b34f44511f2cfb77e3c670f0d: Status 404 returned error can't find the container with id a42e3ec699b3785aaa8c1139d3bacc1c6afa531b34f44511f2cfb77e3c670f0d
Jul 07 00:20:03.782637 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:03.782601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn" event={"ID":"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae","Type":"ContainerStarted","Data":"c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6"}
Jul 07 00:20:03.782806 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:03.782656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn" event={"ID":"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae","Type":"ContainerStarted","Data":"a42e3ec699b3785aaa8c1139d3bacc1c6afa531b34f44511f2cfb77e3c670f0d"}
Jul 07 00:20:03.782806 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:03.782682    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:03.801103 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:03.801021    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn" podStartSLOduration=1.418190453 podStartE2EDuration="1.801007665s" totalImagesPullingTime="382.817212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:03.799288645 +0000 UTC m=+1904.864620636" watchObservedRunningTime="2026-07-07 00:20:03.801007665 +0000 UTC m=+1904.866339642"
Jul 07 00:20:09.792124 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:09.792098    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:15.349044 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.348999    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"]
Jul 07 00:20:15.349590 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.349354    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh" podUID="fa13fa83-6304-48ba-b3ef-0abfe95f1a37" containerName="maas-api" containerID="cri-o://f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e" gracePeriod=30
Jul 07 00:20:15.610355 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.610286    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:20:15.700745 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.700708    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctllb\" (UniqueName: \"kubernetes.io/projected/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-kube-api-access-ctllb\") pod \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") "
Jul 07 00:20:15.700745 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.700751    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-openshift-service-ca\") pod \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") "
Jul 07 00:20:15.700971 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.700819    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls\") pod \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\" (UID: \"fa13fa83-6304-48ba-b3ef-0abfe95f1a37\") "
Jul 07 00:20:15.701174 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.701148    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa13fa83-6304-48ba-b3ef-0abfe95f1a37" (UID: "fa13fa83-6304-48ba-b3ef-0abfe95f1a37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:20:15.703052 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.703026    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-kube-api-access-ctllb" (OuterVolumeSpecName: "kube-api-access-ctllb") pod "fa13fa83-6304-48ba-b3ef-0abfe95f1a37" (UID: "fa13fa83-6304-48ba-b3ef-0abfe95f1a37"). InnerVolumeSpecName "kube-api-access-ctllb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:20:15.703212 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.703195    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa13fa83-6304-48ba-b3ef-0abfe95f1a37" (UID: "fa13fa83-6304-48ba-b3ef-0abfe95f1a37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:20:15.801515 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.801473    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctllb\" (UniqueName: \"kubernetes.io/projected/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-kube-api-access-ctllb\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:20:15.801515 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.801507    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:20:15.801515 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.801517    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa13fa83-6304-48ba-b3ef-0abfe95f1a37-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:20:15.843153 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.843121    2564 generic.go:356] "Generic (PLEG): container finished" podID="fa13fa83-6304-48ba-b3ef-0abfe95f1a37" containerID="f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e" exitCode=0
Jul 07 00:20:15.843330 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.843168    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh" event={"ID":"fa13fa83-6304-48ba-b3ef-0abfe95f1a37","Type":"ContainerDied","Data":"f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e"}
Jul 07 00:20:15.843330 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.843182    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"
Jul 07 00:20:15.843330 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.843212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh" event={"ID":"fa13fa83-6304-48ba-b3ef-0abfe95f1a37","Type":"ContainerDied","Data":"a6ce76fbb20695ccdc4881faec520ea8693b8f63157fdaa26b4dee9d4546a0d1"}
Jul 07 00:20:15.843330 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.843235    2564 scope.go:117] "RemoveContainer" containerID="f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e"
Jul 07 00:20:15.852506 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.852486    2564 scope.go:117] "RemoveContainer" containerID="f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e"
Jul 07 00:20:15.852796 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:15.852775    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e\": container with ID starting with f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e not found: ID does not exist" containerID="f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e"
Jul 07 00:20:15.852848 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.852805    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e"} err="failed to get container status \"f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e\": rpc error: code = NotFound desc = could not find container \"f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e\": container with ID starting with f87bb1fd1bfbda0e1df27aff39917050cde41ed58086e52ea271594770bd7f0e not found: ID does not exist"
Jul 07 00:20:15.865203 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.865121    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"]
Jul 07 00:20:15.869568 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:15.869539    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-4d772e-64b7757dc6-c7tlh"]
Jul 07 00:20:17.391550 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:17.391522    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa13fa83-6304-48ba-b3ef-0abfe95f1a37" path="/var/lib/kubelet/pods/fa13fa83-6304-48ba-b3ef-0abfe95f1a37/volumes"
Jul 07 00:20:23.218327 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.218292    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"]
Jul 07 00:20:23.218870 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.218595    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn" podUID="16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" containerName="maas-api" containerID="cri-o://c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6" gracePeriod=30
Jul 07 00:20:23.467762 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.467729    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:23.566857 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.566753    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-openshift-service-ca\") pod \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") "
Jul 07 00:20:23.566857 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.566843    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls\") pod \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") "
Jul 07 00:20:23.567121 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.566875    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45zs4\" (UniqueName: \"kubernetes.io/projected/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-kube-api-access-45zs4\") pod \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\" (UID: \"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae\") "
Jul 07 00:20:23.567222 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.567195    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" (UID: "16fa43c3-fd13-47bb-8d72-e4d02ddd15ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:20:23.569204 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.569171    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" (UID: "16fa43c3-fd13-47bb-8d72-e4d02ddd15ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:20:23.569319 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.569208    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-kube-api-access-45zs4" (OuterVolumeSpecName: "kube-api-access-45zs4") pod "16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" (UID: "16fa43c3-fd13-47bb-8d72-e4d02ddd15ae"). InnerVolumeSpecName "kube-api-access-45zs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:20:23.667748 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.667707    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:20:23.667748 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.667744    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:20:23.667748 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.667755    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45zs4\" (UniqueName: \"kubernetes.io/projected/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae-kube-api-access-45zs4\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:20:23.880285 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.880197    2564 generic.go:356] "Generic (PLEG): container finished" podID="16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" containerID="c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6" exitCode=0
Jul 07 00:20:23.880285 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.880242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn" event={"ID":"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae","Type":"ContainerDied","Data":"c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6"}
Jul 07 00:20:23.880285 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.880254    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"
Jul 07 00:20:23.880572 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.880314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn" event={"ID":"16fa43c3-fd13-47bb-8d72-e4d02ddd15ae","Type":"ContainerDied","Data":"a42e3ec699b3785aaa8c1139d3bacc1c6afa531b34f44511f2cfb77e3c670f0d"}
Jul 07 00:20:23.880572 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.880335    2564 scope.go:117] "RemoveContainer" containerID="c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6"
Jul 07 00:20:23.889060 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.889038    2564 scope.go:117] "RemoveContainer" containerID="c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6"
Jul 07 00:20:23.889340 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:23.889321    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6\": container with ID starting with c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6 not found: ID does not exist" containerID="c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6"
Jul 07 00:20:23.889406 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.889348    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6"} err="failed to get container status \"c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6\": rpc error: code = NotFound desc = could not find container \"c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6\": container with ID starting with c8ed75e281f695f52076d6f8513c85860fdb07df873fdc4687dca0d42bc6efa6 not found: ID does not exist"
Jul 07 00:20:23.901622 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.901593    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"]
Jul 07 00:20:23.905682 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:23.905657    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5b1463-785b8898f5-fcbdn"]
Jul 07 00:20:25.391241 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:25.391208    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" path="/var/lib/kubelet/pods/16fa43c3-fd13-47bb-8d72-e4d02ddd15ae/volumes"
Jul 07 00:20:32.400134 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.400098    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"]
Jul 07 00:20:32.400630 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.400614    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" containerName="maas-api"
Jul 07 00:20:32.400677 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.400634    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" containerName="maas-api"
Jul 07 00:20:32.400677 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.400651    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa13fa83-6304-48ba-b3ef-0abfe95f1a37" containerName="maas-api"
Jul 07 00:20:32.400677 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.400660    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa13fa83-6304-48ba-b3ef-0abfe95f1a37" containerName="maas-api"
Jul 07 00:20:32.400768 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.400743    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa13fa83-6304-48ba-b3ef-0abfe95f1a37" containerName="maas-api"
Jul 07 00:20:32.400768 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.400757    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="16fa43c3-fd13-47bb-8d72-e4d02ddd15ae" containerName="maas-api"
Jul 07 00:20:32.405233 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.405213    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.407863 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.407845    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-b60cd9-serving-cert\""
Jul 07 00:20:32.414058 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.414037    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"]
Jul 07 00:20:32.447763 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.447735    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.447930 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.447787    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96871ce9-1eaa-47c0-84fe-3570a9430d89-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.447930 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.447841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh8pg\" (UniqueName: \"kubernetes.io/projected/96871ce9-1eaa-47c0-84fe-3570a9430d89-kube-api-access-dh8pg\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.549234 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.549199    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.549392 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.549257    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96871ce9-1eaa-47c0-84fe-3570a9430d89-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.549392 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.549296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh8pg\" (UniqueName: \"kubernetes.io/projected/96871ce9-1eaa-47c0-84fe-3570a9430d89-kube-api-access-dh8pg\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.549392 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:32.549350    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-b60cd9-serving-cert: secret "maas-api-e2e-shared-a-b60cd9-serving-cert" not found
Jul 07 00:20:32.549559 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:32.549417    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls podName:96871ce9-1eaa-47c0-84fe-3570a9430d89 nodeName:}" failed. No retries permitted until 2026-07-07 00:20:33.049398095 +0000 UTC m=+1934.114730051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls") pod "maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd" (UID: "96871ce9-1eaa-47c0-84fe-3570a9430d89") : secret "maas-api-e2e-shared-a-b60cd9-serving-cert" not found
Jul 07 00:20:32.549911 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.549890    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96871ce9-1eaa-47c0-84fe-3570a9430d89-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:32.558671 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:32.558652    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh8pg\" (UniqueName: \"kubernetes.io/projected/96871ce9-1eaa-47c0-84fe-3570a9430d89-kube-api-access-dh8pg\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:33.053047 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:33.053014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:33.055583 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:33.055564    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls\") pod \"maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") " pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:33.316262 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:33.316158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:33.459594 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:33.459518    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"]
Jul 07 00:20:33.461946 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:20:33.461917    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96871ce9_1eaa_47c0_84fe_3570a9430d89.slice/crio-795d552690c86bb51621de64d83633298de0a860b5484b04246f88aea3630463 WatchSource:0}: Error finding container 795d552690c86bb51621de64d83633298de0a860b5484b04246f88aea3630463: Status 404 returned error can't find the container with id 795d552690c86bb51621de64d83633298de0a860b5484b04246f88aea3630463
Jul 07 00:20:33.925915 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:33.925880    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd" event={"ID":"96871ce9-1eaa-47c0-84fe-3570a9430d89","Type":"ContainerStarted","Data":"795d552690c86bb51621de64d83633298de0a860b5484b04246f88aea3630463"}
Jul 07 00:20:34.931716 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:34.931672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd" event={"ID":"96871ce9-1eaa-47c0-84fe-3570a9430d89","Type":"ContainerStarted","Data":"629fb4ee2e1c266a1100c787e4dfce28d7009ed0a1876e4582f13a9bc7735653"}
Jul 07 00:20:34.932145 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:34.931807    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:34.948798 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:34.948750    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd" podStartSLOduration=2.479822406 podStartE2EDuration="2.948736399s" totalImagesPullingTime="468.913993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:34.948363358 +0000 UTC m=+1936.013695336" watchObservedRunningTime="2026-07-07 00:20:34.948736399 +0000 UTC m=+1936.014068376"
Jul 07 00:20:40.587174 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.587137    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"]
Jul 07 00:20:40.591276 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.591249    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.594604 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.594579    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-c6e67e-serving-cert\""
Jul 07 00:20:40.602321 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.602290    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"]
Jul 07 00:20:40.720596 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.720559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.720761 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.720617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bz5n\" (UniqueName: \"kubernetes.io/projected/0066ef9e-45e7-4199-b549-2c6996f8b5bb-kube-api-access-4bz5n\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.720761 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.720673    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ef9e-45e7-4199-b549-2c6996f8b5bb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.821519 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.821478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.821705 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.821534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bz5n\" (UniqueName: \"kubernetes.io/projected/0066ef9e-45e7-4199-b549-2c6996f8b5bb-kube-api-access-4bz5n\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.821705 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:40.821626    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-c6e67e-serving-cert: secret "maas-api-e2e-shared-b-c6e67e-serving-cert" not found
Jul 07 00:20:40.821705 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.821646    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ef9e-45e7-4199-b549-2c6996f8b5bb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.821705 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:20:40.821689    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls podName:0066ef9e-45e7-4199-b549-2c6996f8b5bb nodeName:}" failed. No retries permitted until 2026-07-07 00:20:41.321671998 +0000 UTC m=+1942.387003953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls") pod "maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5" (UID: "0066ef9e-45e7-4199-b549-2c6996f8b5bb") : secret "maas-api-e2e-shared-b-c6e67e-serving-cert" not found
Jul 07 00:20:40.822261 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.822244    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ef9e-45e7-4199-b549-2c6996f8b5bb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.832603 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.832578    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bz5n\" (UniqueName: \"kubernetes.io/projected/0066ef9e-45e7-4199-b549-2c6996f8b5bb-kube-api-access-4bz5n\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:40.942698 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:40.942624    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:20:41.326339 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:41.326300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:41.329132 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:41.329100    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") " pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:41.503543 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:41.503505    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:41.646411 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:41.646176    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"]
Jul 07 00:20:41.648529 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:20:41.648497    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0066ef9e_45e7_4199_b549_2c6996f8b5bb.slice/crio-eae18a2999acd239d9c6f388b3b9e16ce4ebe8b3309612568b0308936157edb9 WatchSource:0}: Error finding container eae18a2999acd239d9c6f388b3b9e16ce4ebe8b3309612568b0308936157edb9: Status 404 returned error can't find the container with id eae18a2999acd239d9c6f388b3b9e16ce4ebe8b3309612568b0308936157edb9
Jul 07 00:20:41.968616 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:41.968579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5" event={"ID":"0066ef9e-45e7-4199-b549-2c6996f8b5bb","Type":"ContainerStarted","Data":"eae18a2999acd239d9c6f388b3b9e16ce4ebe8b3309612568b0308936157edb9"}
Jul 07 00:20:42.976583 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:42.976545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5" event={"ID":"0066ef9e-45e7-4199-b549-2c6996f8b5bb","Type":"ContainerStarted","Data":"f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f"}
Jul 07 00:20:42.977022 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:42.976687    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:20:42.994590 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:42.994538    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5" podStartSLOduration=2.5540559849999998 podStartE2EDuration="2.994520504s" totalImagesPullingTime="440.464519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:42.992770808 +0000 UTC m=+1944.058102785" watchObservedRunningTime="2026-07-07 00:20:42.994520504 +0000 UTC m=+1944.059852481"
Jul 07 00:20:48.987654 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:20:48.987621    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:21:04.508280 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.508244    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"]
Jul 07 00:21:04.511734 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.511713    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.515702 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.515679    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"default-dockercfg-hjltn\""
Jul 07 00:21:04.515810 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.515710    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"openshift-service-ca.crt\""
Jul 07 00:21:04.515810 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.515741    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"auth-test-model-b60cd9-kserve-self-signed-certs\""
Jul 07 00:21:04.515810 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.515792    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"kube-root-ca.crt\""
Jul 07 00:21:04.521748 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.521724    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"]
Jul 07 00:21:04.529491 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.529471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-tmp-dir\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.529612 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.529497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmp9k\" (UniqueName: \"kubernetes.io/projected/3286bce2-1159-4757-ba48-d5dbba5ac755-kube-api-access-cmp9k\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.529612 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.529533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-home\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.529733 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.529653    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3286bce2-1159-4757-ba48-d5dbba5ac755-tls-certs\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.529733 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.529706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-model-cache\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.529839 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.529788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-dshm\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631181 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631151    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-home\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631327 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3286bce2-1159-4757-ba48-d5dbba5ac755-tls-certs\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631327 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-model-cache\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631327 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-dshm\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631327 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631304    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-tmp-dir\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631521 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmp9k\" (UniqueName: \"kubernetes.io/projected/3286bce2-1159-4757-ba48-d5dbba5ac755-kube-api-access-cmp9k\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631614 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631591    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-home\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631660 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631625    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-model-cache\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.631698 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.631672    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-tmp-dir\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.633654 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.633624    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-dshm\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.634026 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.634007    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3286bce2-1159-4757-ba48-d5dbba5ac755-tls-certs\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.643364 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.643342    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmp9k\" (UniqueName: \"kubernetes.io/projected/3286bce2-1159-4757-ba48-d5dbba5ac755-kube-api-access-cmp9k\") pod \"auth-test-model-b60cd9-kserve-785b45568c-5p9lb\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") " pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.821960 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.821874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:04.966230 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:04.966160    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"]
Jul 07 00:21:04.968540 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:21:04.968502    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3286bce2_1159_4757_ba48_d5dbba5ac755.slice/crio-f1a6047fca4041b1bee612f8c45ff915fb6beee70d5098bafea4c9ea0f28fcc4 WatchSource:0}: Error finding container f1a6047fca4041b1bee612f8c45ff915fb6beee70d5098bafea4c9ea0f28fcc4: Status 404 returned error can't find the container with id f1a6047fca4041b1bee612f8c45ff915fb6beee70d5098bafea4c9ea0f28fcc4
Jul 07 00:21:05.075695 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:05.075615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb" event={"ID":"3286bce2-1159-4757-ba48-d5dbba5ac755","Type":"ContainerStarted","Data":"a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf"}
Jul 07 00:21:05.075695 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:05.075662    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb" event={"ID":"3286bce2-1159-4757-ba48-d5dbba5ac755","Type":"ContainerStarted","Data":"f1a6047fca4041b1bee612f8c45ff915fb6beee70d5098bafea4c9ea0f28fcc4"}
Jul 07 00:21:05.096967 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:05.096913    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb" podStartSLOduration=1.096898887 podStartE2EDuration="1.096898887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:05.095337655 +0000 UTC m=+1966.160669629" watchObservedRunningTime="2026-07-07 00:21:05.096898887 +0000 UTC m=+1966.162230916"
Jul 07 00:21:14.822473 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:14.822371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:14.822921 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:14.822653    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:14.835735 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:14.835712    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:15.132088 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:15.132001    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:21:25.939686 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:25.939638    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"]
Jul 07 00:21:25.943265 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:25.943243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:25.948660 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:25.948632    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"default-dockercfg-qqmkr\""
Jul 07 00:21:25.948812 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:25.948632    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"kube-root-ca.crt\""
Jul 07 00:21:25.948812 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:25.948680    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"openshift-service-ca.crt\""
Jul 07 00:21:25.948812 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:25.948689    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"auth-test-model-c6e67e-kserve-self-signed-certs\""
Jul 07 00:21:25.955210 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:25.955185    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"]
Jul 07 00:21:26.025986 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.025950    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-model-cache\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.026161 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.025997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-home\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.026161 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.026036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tls-certs\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.026161 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.026059    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tmp-dir\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.026161 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.026088    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjgt5\" (UniqueName: \"kubernetes.io/projected/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-kube-api-access-bjgt5\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.026161 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.026124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-dshm\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127173 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tls-certs\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127173 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tmp-dir\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127478 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjgt5\" (UniqueName: \"kubernetes.io/projected/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-kube-api-access-bjgt5\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127478 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127248    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-dshm\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127478 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-model-cache\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127478 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-home\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127695 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127638    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tmp-dir\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127780 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127755    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-home\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.127818 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.127767    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-model-cache\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.129840 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.129811    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-dshm\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.129951 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.129891    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tls-certs\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.138710 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.138683    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjgt5\" (UniqueName: \"kubernetes.io/projected/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-kube-api-access-bjgt5\") pod \"auth-test-model-c6e67e-kserve-696bd54b7f-8nq89\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") " pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.253070 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.253028    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:26.407750 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:26.407723    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"]
Jul 07 00:21:26.410308 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:21:26.410281    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe73c3be_4ce5_48c3_8342_2b1cb4ba4a84.slice/crio-1b95fbc6236ab5ccaa34d9197c18b007fb315280181c18873571fd5e4146339e WatchSource:0}: Error finding container 1b95fbc6236ab5ccaa34d9197c18b007fb315280181c18873571fd5e4146339e: Status 404 returned error can't find the container with id 1b95fbc6236ab5ccaa34d9197c18b007fb315280181c18873571fd5e4146339e
Jul 07 00:21:27.175774 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:27.175738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89" event={"ID":"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84","Type":"ContainerStarted","Data":"6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d"}
Jul 07 00:21:27.176173 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:27.175782    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89" event={"ID":"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84","Type":"ContainerStarted","Data":"1b95fbc6236ab5ccaa34d9197c18b007fb315280181c18873571fd5e4146339e"}
Jul 07 00:21:27.198180 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:27.198124    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89" podStartSLOduration=2.198107473 podStartE2EDuration="2.198107473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:27.19585202 +0000 UTC m=+1988.261183996" watchObservedRunningTime="2026-07-07 00:21:27.198107473 +0000 UTC m=+1988.263439451"
Jul 07 00:21:36.253324 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:36.253275    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:36.253324 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:36.253338    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:36.265984 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:36.265956    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:21:37.240461 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:21:37.240418    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:23:19.431569 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:19.431540    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:23:19.435480 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:19.435458    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s9mv7_70c9a419-4f23-46fa-be19-bc75d9d97011/ovn-acl-logging/0.log"
Jul 07 00:23:23.393375 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:23.393347    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"]
Jul 07 00:23:24.472673 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:24.472639    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"]
Jul 07 00:23:38.413864 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.413820    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb" podUID="3286bce2-1159-4757-ba48-d5dbba5ac755" containerName="main" containerID="cri-o://a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf" gracePeriod=45
Jul 07 00:23:38.414972 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.414937    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"]
Jul 07 00:23:38.418358 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.418340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.421285 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.421266    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"sub-test-model-b60cd9-kserve-self-signed-certs\""
Jul 07 00:23:38.427219 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.427196    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"]
Jul 07 00:23:38.514223 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.514192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-home\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.514395 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.514227    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-dshm\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.514395 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.514250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-tmp-dir\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.514395 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.514346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98deef27-d7bb-4beb-a4c1-064a29e91c13-tls-certs\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.514528 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.514448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqxd\" (UniqueName: \"kubernetes.io/projected/98deef27-d7bb-4beb-a4c1-064a29e91c13-kube-api-access-nzqxd\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.514528 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.514506    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-model-cache\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615149 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615118    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-model-cache\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615313 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-home\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615313 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-dshm\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615313 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-tmp-dir\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615313 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98deef27-d7bb-4beb-a4c1-064a29e91c13-tls-certs\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615569 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqxd\" (UniqueName: \"kubernetes.io/projected/98deef27-d7bb-4beb-a4c1-064a29e91c13-kube-api-access-nzqxd\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615622 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615598    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-home\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615622 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615613    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-tmp-dir\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.615720 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.615683    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-model-cache\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.617923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.617897    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-dshm\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.618362 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.618342    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98deef27-d7bb-4beb-a4c1-064a29e91c13-tls-certs\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.624714 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.624689    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqxd\" (UniqueName: \"kubernetes.io/projected/98deef27-d7bb-4beb-a4c1-064a29e91c13-kube-api-access-nzqxd\") pod \"sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") " pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.676358 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.676289    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:23:38.716730 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.716690    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmp9k\" (UniqueName: \"kubernetes.io/projected/3286bce2-1159-4757-ba48-d5dbba5ac755-kube-api-access-cmp9k\") pod \"3286bce2-1159-4757-ba48-d5dbba5ac755\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") "
Jul 07 00:23:38.716923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.716758    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-dshm\") pod \"3286bce2-1159-4757-ba48-d5dbba5ac755\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") "
Jul 07 00:23:38.716923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.716782    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-tmp-dir\") pod \"3286bce2-1159-4757-ba48-d5dbba5ac755\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") "
Jul 07 00:23:38.716923 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.716881    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3286bce2-1159-4757-ba48-d5dbba5ac755-tls-certs\") pod \"3286bce2-1159-4757-ba48-d5dbba5ac755\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") "
Jul 07 00:23:38.717114 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.716966    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-model-cache\") pod \"3286bce2-1159-4757-ba48-d5dbba5ac755\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") "
Jul 07 00:23:38.717114 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.717005    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-home\") pod \"3286bce2-1159-4757-ba48-d5dbba5ac755\" (UID: \"3286bce2-1159-4757-ba48-d5dbba5ac755\") "
Jul 07 00:23:38.717114 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.717020    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3286bce2-1159-4757-ba48-d5dbba5ac755" (UID: "3286bce2-1159-4757-ba48-d5dbba5ac755"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:38.717327 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.717306    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:38.717327 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.717313    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-model-cache" (OuterVolumeSpecName: "model-cache") pod "3286bce2-1159-4757-ba48-d5dbba5ac755" (UID: "3286bce2-1159-4757-ba48-d5dbba5ac755"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:38.717469 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.717328    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-home" (OuterVolumeSpecName: "home") pod "3286bce2-1159-4757-ba48-d5dbba5ac755" (UID: "3286bce2-1159-4757-ba48-d5dbba5ac755"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:38.719219 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.719193    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3286bce2-1159-4757-ba48-d5dbba5ac755-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3286bce2-1159-4757-ba48-d5dbba5ac755" (UID: "3286bce2-1159-4757-ba48-d5dbba5ac755"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:23:38.719497 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.719476    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-dshm" (OuterVolumeSpecName: "dshm") pod "3286bce2-1159-4757-ba48-d5dbba5ac755" (UID: "3286bce2-1159-4757-ba48-d5dbba5ac755"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:38.719586 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.719503    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3286bce2-1159-4757-ba48-d5dbba5ac755-kube-api-access-cmp9k" (OuterVolumeSpecName: "kube-api-access-cmp9k") pod "3286bce2-1159-4757-ba48-d5dbba5ac755" (UID: "3286bce2-1159-4757-ba48-d5dbba5ac755"). InnerVolumeSpecName "kube-api-access-cmp9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:23:38.782505 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.782466    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:38.796997 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.796959    2564 generic.go:356] "Generic (PLEG): container finished" podID="3286bce2-1159-4757-ba48-d5dbba5ac755" containerID="a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf" exitCode=0
Jul 07 00:23:38.797130 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.797012    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"
Jul 07 00:23:38.797130 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.797018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb" event={"ID":"3286bce2-1159-4757-ba48-d5dbba5ac755","Type":"ContainerDied","Data":"a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf"}
Jul 07 00:23:38.797130 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.797069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb" event={"ID":"3286bce2-1159-4757-ba48-d5dbba5ac755","Type":"ContainerDied","Data":"f1a6047fca4041b1bee612f8c45ff915fb6beee70d5098bafea4c9ea0f28fcc4"}
Jul 07 00:23:38.797130 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.797092    2564 scope.go:117] "RemoveContainer" containerID="a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf"
Jul 07 00:23:38.806525 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.806503    2564 scope.go:117] "RemoveContainer" containerID="a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf"
Jul 07 00:23:38.806835 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:23:38.806802    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf\": container with ID starting with a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf not found: ID does not exist" containerID="a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf"
Jul 07 00:23:38.806904 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.806854    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf"} err="failed to get container status \"a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf\": rpc error: code = NotFound desc = could not find container \"a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf\": container with ID starting with a304053ca7d5acd43c6ae189533e42dacdc60ed9369ba9dedf9aa67c5ae9fddf not found: ID does not exist"
Jul 07 00:23:38.818357 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.818326    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:38.818514 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.818359    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmp9k\" (UniqueName: \"kubernetes.io/projected/3286bce2-1159-4757-ba48-d5dbba5ac755-kube-api-access-cmp9k\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:38.818514 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.818375    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:38.818514 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.818389    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3286bce2-1159-4757-ba48-d5dbba5ac755-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:38.818514 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.818404    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3286bce2-1159-4757-ba48-d5dbba5ac755-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:38.820907 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.820882    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"]
Jul 07 00:23:38.826052 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.826026    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/auth-test-model-b60cd9-kserve-785b45568c-5p9lb"]
Jul 07 00:23:38.925302 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:38.925269    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"]
Jul 07 00:23:38.927230 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:23:38.927176    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98deef27_d7bb_4beb_a4c1_064a29e91c13.slice/crio-b9a0e419db0236962dd0dac84b30bcd18d70247b50fb898d9527169ad34c8228 WatchSource:0}: Error finding container b9a0e419db0236962dd0dac84b30bcd18d70247b50fb898d9527169ad34c8228: Status 404 returned error can't find the container with id b9a0e419db0236962dd0dac84b30bcd18d70247b50fb898d9527169ad34c8228
Jul 07 00:23:39.390779 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.390748    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3286bce2-1159-4757-ba48-d5dbba5ac755" path="/var/lib/kubelet/pods/3286bce2-1159-4757-ba48-d5dbba5ac755/volumes"
Jul 07 00:23:39.493402 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.493357    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89" podUID="be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" containerName="main" containerID="cri-o://6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d" gracePeriod=45
Jul 07 00:23:39.743905 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.743868    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:23:39.803577 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.803545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh" event={"ID":"98deef27-d7bb-4beb-a4c1-064a29e91c13","Type":"ContainerStarted","Data":"fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121"}
Jul 07 00:23:39.803783 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.803590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh" event={"ID":"98deef27-d7bb-4beb-a4c1-064a29e91c13","Type":"ContainerStarted","Data":"b9a0e419db0236962dd0dac84b30bcd18d70247b50fb898d9527169ad34c8228"}
Jul 07 00:23:39.805647 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.805619    2564 generic.go:356] "Generic (PLEG): container finished" podID="be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" containerID="6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d" exitCode=0
Jul 07 00:23:39.805762 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.805661    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"
Jul 07 00:23:39.805762 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.805675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89" event={"ID":"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84","Type":"ContainerDied","Data":"6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d"}
Jul 07 00:23:39.805762 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.805704    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89" event={"ID":"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84","Type":"ContainerDied","Data":"1b95fbc6236ab5ccaa34d9197c18b007fb315280181c18873571fd5e4146339e"}
Jul 07 00:23:39.805762 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.805724    2564 scope.go:117] "RemoveContainer" containerID="6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d"
Jul 07 00:23:39.814105 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.814085    2564 scope.go:117] "RemoveContainer" containerID="6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d"
Jul 07 00:23:39.814395 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:23:39.814375    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d\": container with ID starting with 6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d not found: ID does not exist" containerID="6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d"
Jul 07 00:23:39.814487 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.814405    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d"} err="failed to get container status \"6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d\": rpc error: code = NotFound desc = could not find container \"6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d\": container with ID starting with 6633a17716bfa9c8436d125d3527a9e4213c924bb8f21a9b5dd131b3e427af0d not found: ID does not exist"
Jul 07 00:23:39.823529 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.823482    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh" podStartSLOduration=1.823465895 podStartE2EDuration="1.823465895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:39.82213247 +0000 UTC m=+2120.887464447" watchObservedRunningTime="2026-07-07 00:23:39.823465895 +0000 UTC m=+2120.888797869"
Jul 07 00:23:39.826234 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-model-cache\") pod \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") "
Jul 07 00:23:39.826345 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826248    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjgt5\" (UniqueName: \"kubernetes.io/projected/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-kube-api-access-bjgt5\") pod \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") "
Jul 07 00:23:39.826345 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-home\") pod \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") "
Jul 07 00:23:39.826483 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826364    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-dshm\") pod \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") "
Jul 07 00:23:39.826483 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826401    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tmp-dir\") pod \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") "
Jul 07 00:23:39.826565 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826491    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tls-certs\") pod \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\" (UID: \"be73c3be-4ce5-48c3-8342-2b1cb4ba4a84\") "
Jul 07 00:23:39.826565 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826497    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-model-cache" (OuterVolumeSpecName: "model-cache") pod "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" (UID: "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:39.826656 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826613    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-home" (OuterVolumeSpecName: "home") pod "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" (UID: "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:39.826748 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826728    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" (UID: "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:39.826849 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826793    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:39.826849 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.826843    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:39.828870 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.828839    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-dshm" (OuterVolumeSpecName: "dshm") pod "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" (UID: "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:23:39.828961 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.828884    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" (UID: "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:23:39.829008 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.828980    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-kube-api-access-bjgt5" (OuterVolumeSpecName: "kube-api-access-bjgt5") pod "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" (UID: "be73c3be-4ce5-48c3-8342-2b1cb4ba4a84"). InnerVolumeSpecName "kube-api-access-bjgt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:23:39.927844 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.927746    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:39.927844 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.927779    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:39.927844 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.927788    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:39.927844 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:39.927798    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjgt5\" (UniqueName: \"kubernetes.io/projected/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84-kube-api-access-bjgt5\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:23:40.129217 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:40.129181    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"]
Jul 07 00:23:40.136830 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:40.136789    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/auth-test-model-c6e67e-kserve-696bd54b7f-8nq89"]
Jul 07 00:23:41.392236 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:41.392194    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" path="/var/lib/kubelet/pods/be73c3be-4ce5-48c3-8342-2b1cb4ba4a84/volumes"
Jul 07 00:23:48.782813 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:48.782782    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:48.783289 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:48.782849    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:48.795020 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:48.794994    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:23:48.862123 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:23:48.862098    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:24:02.628866 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.628817    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"]
Jul 07 00:24:02.629272 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.629198    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3286bce2-1159-4757-ba48-d5dbba5ac755" containerName="main"
Jul 07 00:24:02.629272 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.629209    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3286bce2-1159-4757-ba48-d5dbba5ac755" containerName="main"
Jul 07 00:24:02.629272 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.629222    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" containerName="main"
Jul 07 00:24:02.629272 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.629227    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" containerName="main"
Jul 07 00:24:02.629405 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.629293    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3286bce2-1159-4757-ba48-d5dbba5ac755" containerName="main"
Jul 07 00:24:02.629405 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.629302    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="be73c3be-4ce5-48c3-8342-2b1cb4ba4a84" containerName="main"
Jul 07 00:24:02.632143 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.632126    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.636687 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.636663    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"openshift-service-ca.crt\""
Jul 07 00:24:02.636687 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.636678    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"kube-root-ca.crt\""
Jul 07 00:24:02.637026 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.637009    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"sub-test-model-c6e67e-kserve-self-signed-certs\""
Jul 07 00:24:02.637104 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.637014    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"default-dockercfg-qqmkr\""
Jul 07 00:24:02.645815 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.645792    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"]
Jul 07 00:24:02.724978 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.724949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb7154f-c854-416e-8489-73a39944260c-tls-certs\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.725150 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.725000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-home\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.725150 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.725081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-tmp-dir\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.725150 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.725125    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-dshm\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.725150 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.725147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-model-cache\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.725305 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.725172    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcwrk\" (UniqueName: \"kubernetes.io/projected/cfb7154f-c854-416e-8489-73a39944260c-kube-api-access-xcwrk\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.826534 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.826494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-dshm\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.826534 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.826539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-model-cache\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.826831 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.826568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcwrk\" (UniqueName: \"kubernetes.io/projected/cfb7154f-c854-416e-8489-73a39944260c-kube-api-access-xcwrk\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.826831 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.826600    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb7154f-c854-416e-8489-73a39944260c-tls-certs\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.826831 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.826656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-home\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.826831 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.826731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-tmp-dir\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.827081 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.827056    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-model-cache\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.827143 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.827096    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-tmp-dir\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.827143 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.827113    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-home\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.828972 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.828948    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-dshm\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.829144 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.829123    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb7154f-c854-416e-8489-73a39944260c-tls-certs\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.836051 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.836026    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcwrk\" (UniqueName: \"kubernetes.io/projected/cfb7154f-c854-416e-8489-73a39944260c-kube-api-access-xcwrk\") pod \"sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") " pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:02.941418 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:02.941330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:03.077230 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:03.077185    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"]
Jul 07 00:24:03.079650 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:24:03.079625    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfb7154f_c854_416e_8489_73a39944260c.slice/crio-bba4a1ad5749a8b60b7eac79799b9a1060c2df6f378d67bbde186e81feeb2032 WatchSource:0}: Error finding container bba4a1ad5749a8b60b7eac79799b9a1060c2df6f378d67bbde186e81feeb2032: Status 404 returned error can't find the container with id bba4a1ad5749a8b60b7eac79799b9a1060c2df6f378d67bbde186e81feeb2032
Jul 07 00:24:03.926988 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:03.926946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh" event={"ID":"cfb7154f-c854-416e-8489-73a39944260c","Type":"ContainerStarted","Data":"ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5"}
Jul 07 00:24:03.927374 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:03.926995    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh" event={"ID":"cfb7154f-c854-416e-8489-73a39944260c","Type":"ContainerStarted","Data":"bba4a1ad5749a8b60b7eac79799b9a1060c2df6f378d67bbde186e81feeb2032"}
Jul 07 00:24:03.951218 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:03.951170    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh" podStartSLOduration=1.951154996 podStartE2EDuration="1.951154996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:03.948966908 +0000 UTC m=+2145.014298895" watchObservedRunningTime="2026-07-07 00:24:03.951154996 +0000 UTC m=+2145.016486972"
Jul 07 00:24:12.942475 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:12.942361    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:12.942475 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:12.942435    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:12.955542 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:12.955520    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:24:12.983951 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:24:12.983929    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:25:01.817407 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:01.817366    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"]
Jul 07 00:25:02.684243 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:02.684209    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"]
Jul 07 00:25:16.838897 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:16.838858    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh" podUID="98deef27-d7bb-4beb-a4c1-064a29e91c13" containerName="main" containerID="cri-o://fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121" gracePeriod=45
Jul 07 00:25:17.086753 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.086721    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:25:17.184287 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184212    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-tmp-dir\") pod \"98deef27-d7bb-4beb-a4c1-064a29e91c13\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") "
Jul 07 00:25:17.184287 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184251    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-model-cache\") pod \"98deef27-d7bb-4beb-a4c1-064a29e91c13\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") "
Jul 07 00:25:17.184513 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184305    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzqxd\" (UniqueName: \"kubernetes.io/projected/98deef27-d7bb-4beb-a4c1-064a29e91c13-kube-api-access-nzqxd\") pod \"98deef27-d7bb-4beb-a4c1-064a29e91c13\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") "
Jul 07 00:25:17.184513 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184340    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-home\") pod \"98deef27-d7bb-4beb-a4c1-064a29e91c13\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") "
Jul 07 00:25:17.184513 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184382    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98deef27-d7bb-4beb-a4c1-064a29e91c13-tls-certs\") pod \"98deef27-d7bb-4beb-a4c1-064a29e91c13\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") "
Jul 07 00:25:17.184513 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184465    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-dshm\") pod \"98deef27-d7bb-4beb-a4c1-064a29e91c13\" (UID: \"98deef27-d7bb-4beb-a4c1-064a29e91c13\") "
Jul 07 00:25:17.184730 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184561    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98deef27-d7bb-4beb-a4c1-064a29e91c13" (UID: "98deef27-d7bb-4beb-a4c1-064a29e91c13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:17.184730 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184606    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-home" (OuterVolumeSpecName: "home") pod "98deef27-d7bb-4beb-a4c1-064a29e91c13" (UID: "98deef27-d7bb-4beb-a4c1-064a29e91c13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:17.184730 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184640    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-model-cache" (OuterVolumeSpecName: "model-cache") pod "98deef27-d7bb-4beb-a4c1-064a29e91c13" (UID: "98deef27-d7bb-4beb-a4c1-064a29e91c13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:17.184856 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184812    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:17.184856 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184829    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:17.184856 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.184839    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:17.186613 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.186586    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98deef27-d7bb-4beb-a4c1-064a29e91c13-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98deef27-d7bb-4beb-a4c1-064a29e91c13" (UID: "98deef27-d7bb-4beb-a4c1-064a29e91c13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:25:17.187049 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.187021    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-dshm" (OuterVolumeSpecName: "dshm") pod "98deef27-d7bb-4beb-a4c1-064a29e91c13" (UID: "98deef27-d7bb-4beb-a4c1-064a29e91c13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:17.187137 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.187062    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98deef27-d7bb-4beb-a4c1-064a29e91c13-kube-api-access-nzqxd" (OuterVolumeSpecName: "kube-api-access-nzqxd") pod "98deef27-d7bb-4beb-a4c1-064a29e91c13" (UID: "98deef27-d7bb-4beb-a4c1-064a29e91c13"). InnerVolumeSpecName "kube-api-access-nzqxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:25:17.271718 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.271688    2564 generic.go:356] "Generic (PLEG): container finished" podID="98deef27-d7bb-4beb-a4c1-064a29e91c13" containerID="fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121" exitCode=0
Jul 07 00:25:17.271840 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.271731    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"
Jul 07 00:25:17.271840 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.271766    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh" event={"ID":"98deef27-d7bb-4beb-a4c1-064a29e91c13","Type":"ContainerDied","Data":"fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121"}
Jul 07 00:25:17.271840 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.271810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh" event={"ID":"98deef27-d7bb-4beb-a4c1-064a29e91c13","Type":"ContainerDied","Data":"b9a0e419db0236962dd0dac84b30bcd18d70247b50fb898d9527169ad34c8228"}
Jul 07 00:25:17.271840 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.271829    2564 scope.go:117] "RemoveContainer" containerID="fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121"
Jul 07 00:25:17.281947 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.281925    2564 scope.go:117] "RemoveContainer" containerID="fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121"
Jul 07 00:25:17.282497 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:25:17.282468    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121\": container with ID starting with fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121 not found: ID does not exist" containerID="fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121"
Jul 07 00:25:17.282602 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.282505    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121"} err="failed to get container status \"fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121\": rpc error: code = NotFound desc = could not find container \"fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121\": container with ID starting with fe4c4543a530ce68210d2d83c6ef83ca8f96012218e991ef49fad5d705962121 not found: ID does not exist"
Jul 07 00:25:17.285872 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.285850    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98deef27-d7bb-4beb-a4c1-064a29e91c13-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:17.285969 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.285899    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzqxd\" (UniqueName: \"kubernetes.io/projected/98deef27-d7bb-4beb-a4c1-064a29e91c13-kube-api-access-nzqxd\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:17.285969 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.285913    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98deef27-d7bb-4beb-a4c1-064a29e91c13-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:17.298893 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.298864    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"]
Jul 07 00:25:17.303056 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.303031    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/sub-test-model-b60cd9-kserve-54b676b6cf-q6dfh"]
Jul 07 00:25:17.390713 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.390669    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98deef27-d7bb-4beb-a4c1-064a29e91c13" path="/var/lib/kubelet/pods/98deef27-d7bb-4beb-a4c1-064a29e91c13/volumes"
Jul 07 00:25:17.704978 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.704927    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh" podUID="cfb7154f-c854-416e-8489-73a39944260c" containerName="main" containerID="cri-o://ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5" gracePeriod=45
Jul 07 00:25:17.963336 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:17.963272    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:25:18.092155 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092125    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-home\") pod \"cfb7154f-c854-416e-8489-73a39944260c\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") "
Jul 07 00:25:18.092155 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092158    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-tmp-dir\") pod \"cfb7154f-c854-416e-8489-73a39944260c\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") "
Jul 07 00:25:18.092454 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092175    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-dshm\") pod \"cfb7154f-c854-416e-8489-73a39944260c\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") "
Jul 07 00:25:18.092454 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092219    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcwrk\" (UniqueName: \"kubernetes.io/projected/cfb7154f-c854-416e-8489-73a39944260c-kube-api-access-xcwrk\") pod \"cfb7154f-c854-416e-8489-73a39944260c\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") "
Jul 07 00:25:18.092454 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092283    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-model-cache\") pod \"cfb7154f-c854-416e-8489-73a39944260c\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") "
Jul 07 00:25:18.092454 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb7154f-c854-416e-8489-73a39944260c-tls-certs\") pod \"cfb7154f-c854-416e-8489-73a39944260c\" (UID: \"cfb7154f-c854-416e-8489-73a39944260c\") "
Jul 07 00:25:18.092675 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092458    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfb7154f-c854-416e-8489-73a39944260c" (UID: "cfb7154f-c854-416e-8489-73a39944260c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:18.092675 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092495    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-home" (OuterVolumeSpecName: "home") pod "cfb7154f-c854-416e-8489-73a39944260c" (UID: "cfb7154f-c854-416e-8489-73a39944260c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:18.092675 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092612    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfb7154f-c854-416e-8489-73a39944260c" (UID: "cfb7154f-c854-416e-8489-73a39944260c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:18.092819 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092691    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:18.092819 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092713    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:18.092819 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.092728    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:18.094579 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.094554    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-dshm" (OuterVolumeSpecName: "dshm") pod "cfb7154f-c854-416e-8489-73a39944260c" (UID: "cfb7154f-c854-416e-8489-73a39944260c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:18.094579 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.094561    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb7154f-c854-416e-8489-73a39944260c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfb7154f-c854-416e-8489-73a39944260c" (UID: "cfb7154f-c854-416e-8489-73a39944260c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:25:18.094741 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.094573    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb7154f-c854-416e-8489-73a39944260c-kube-api-access-xcwrk" (OuterVolumeSpecName: "kube-api-access-xcwrk") pod "cfb7154f-c854-416e-8489-73a39944260c" (UID: "cfb7154f-c854-416e-8489-73a39944260c"). InnerVolumeSpecName "kube-api-access-xcwrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:25:18.193410 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.193373    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb7154f-c854-416e-8489-73a39944260c-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:18.193410 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.193403    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfb7154f-c854-416e-8489-73a39944260c-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:18.193640 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.193415    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcwrk\" (UniqueName: \"kubernetes.io/projected/cfb7154f-c854-416e-8489-73a39944260c-kube-api-access-xcwrk\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:25:18.278295 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.278258    2564 generic.go:356] "Generic (PLEG): container finished" podID="cfb7154f-c854-416e-8489-73a39944260c" containerID="ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5" exitCode=0
Jul 07 00:25:18.278486 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.278308    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"
Jul 07 00:25:18.278486 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.278336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh" event={"ID":"cfb7154f-c854-416e-8489-73a39944260c","Type":"ContainerDied","Data":"ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5"}
Jul 07 00:25:18.278486 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.278374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh" event={"ID":"cfb7154f-c854-416e-8489-73a39944260c","Type":"ContainerDied","Data":"bba4a1ad5749a8b60b7eac79799b9a1060c2df6f378d67bbde186e81feeb2032"}
Jul 07 00:25:18.278486 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.278397    2564 scope.go:117] "RemoveContainer" containerID="ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5"
Jul 07 00:25:18.287722 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.287704    2564 scope.go:117] "RemoveContainer" containerID="ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5"
Jul 07 00:25:18.288000 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:25:18.287980    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5\": container with ID starting with ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5 not found: ID does not exist" containerID="ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5"
Jul 07 00:25:18.288046 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.288009    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5"} err="failed to get container status \"ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5\": rpc error: code = NotFound desc = could not find container \"ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5\": container with ID starting with ac6563d72850e78ed55d20b18ac14c2458eb8bd8f8e8eb1b6afabf5c83beb6f5 not found: ID does not exist"
Jul 07 00:25:18.299502 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.299472    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"]
Jul 07 00:25:18.303989 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:18.303965    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/sub-test-model-c6e67e-kserve-5fd8476fc9-49zjh"]
Jul 07 00:25:19.392682 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:19.392646    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfb7154f-c854-416e-8489-73a39944260c" path="/var/lib/kubelet/pods/cfb7154f-c854-416e-8489-73a39944260c/volumes"
Jul 07 00:25:21.894675 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.894639    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"]
Jul 07 00:25:21.895169 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.895153    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfb7154f-c854-416e-8489-73a39944260c" containerName="main"
Jul 07 00:25:21.895216 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.895172    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfb7154f-c854-416e-8489-73a39944260c" containerName="main"
Jul 07 00:25:21.895254 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.895226    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98deef27-d7bb-4beb-a4c1-064a29e91c13" containerName="main"
Jul 07 00:25:21.895254 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.895235    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="98deef27-d7bb-4beb-a4c1-064a29e91c13" containerName="main"
Jul 07 00:25:21.895456 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.895334    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfb7154f-c854-416e-8489-73a39944260c" containerName="main"
Jul 07 00:25:21.895456 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.895393    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="98deef27-d7bb-4beb-a4c1-064a29e91c13" containerName="main"
Jul 07 00:25:21.898660 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.898642    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:21.902368 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.902347    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"default-dockercfg-hjltn\""
Jul 07 00:25:21.902368 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.902363    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"kube-root-ca.crt\""
Jul 07 00:25:21.902572 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.902402    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"openshift-service-ca.crt\""
Jul 07 00:25:21.902572 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.902363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b60cd9\"/\"rate-test-model-b60cd9-kserve-self-signed-certs\""
Jul 07 00:25:21.908166 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:21.908142    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"]
Jul 07 00:25:22.029667 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.029626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-dshm\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.029847 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.029707    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-home\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.029847 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.029731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-tmp-dir\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.029847 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.029753    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tdpz\" (UniqueName: \"kubernetes.io/projected/29d48968-b00c-4038-b0b9-f2afb99403b7-kube-api-access-2tdpz\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.029847 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.029815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-model-cache\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.029847 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.029845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d48968-b00c-4038-b0b9-f2afb99403b7-tls-certs\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131219 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-dshm\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131404 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131252    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-home\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131404 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-tmp-dir\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131404 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdpz\" (UniqueName: \"kubernetes.io/projected/29d48968-b00c-4038-b0b9-f2afb99403b7-kube-api-access-2tdpz\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131404 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-model-cache\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131676 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d48968-b00c-4038-b0b9-f2afb99403b7-tls-certs\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131782 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131762    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-model-cache\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131825 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131779    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-tmp-dir\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.131825 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.131806    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-home\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.133856 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.133832    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-dshm\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.134159 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.134139    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d48968-b00c-4038-b0b9-f2afb99403b7-tls-certs\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.139284 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.139261    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tdpz\" (UniqueName: \"kubernetes.io/projected/29d48968-b00c-4038-b0b9-f2afb99403b7-kube-api-access-2tdpz\") pod \"rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") " pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.208493 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.208379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:22.373849 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:22.373816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"]
Jul 07 00:25:22.376606 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:25:22.376581    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29d48968_b00c_4038_b0b9_f2afb99403b7.slice/crio-a32c086e56ea21aeee809ed97de6b75d184a5f729da56b52248720dfa1b22b69 WatchSource:0}: Error finding container a32c086e56ea21aeee809ed97de6b75d184a5f729da56b52248720dfa1b22b69: Status 404 returned error can't find the container with id a32c086e56ea21aeee809ed97de6b75d184a5f729da56b52248720dfa1b22b69
Jul 07 00:25:23.309713 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:23.309674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9" event={"ID":"29d48968-b00c-4038-b0b9-f2afb99403b7","Type":"ContainerStarted","Data":"5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c"}
Jul 07 00:25:23.309713 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:23.309712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9" event={"ID":"29d48968-b00c-4038-b0b9-f2afb99403b7","Type":"ContainerStarted","Data":"a32c086e56ea21aeee809ed97de6b75d184a5f729da56b52248720dfa1b22b69"}
Jul 07 00:25:23.328572 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:23.328520    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9" podStartSLOduration=2.328503845 podStartE2EDuration="2.328503845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:25:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:25:23.327086547 +0000 UTC m=+2224.392418555" watchObservedRunningTime="2026-07-07 00:25:23.328503845 +0000 UTC m=+2224.393835821"
Jul 07 00:25:32.209309 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:32.209271    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:32.209738 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:32.209325    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:32.221692 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:32.221666    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:32.361866 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:32.361838    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:25:48.004889 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.004807    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"]
Jul 07 00:25:48.008324 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.008302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.011352 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.011326    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"kube-root-ca.crt\""
Jul 07 00:25:48.012610 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.012589    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"default-dockercfg-qqmkr\""
Jul 07 00:25:48.012723 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.012619    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"openshift-service-ca.crt\""
Jul 07 00:25:48.012723 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.012640    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6e67e\"/\"rate-test-model-c6e67e-kserve-self-signed-certs\""
Jul 07 00:25:48.028249 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.028217    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"]
Jul 07 00:25:48.075245 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.075223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-model-cache\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.075368 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.075264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-home\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.075368 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.075300    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tmp-dir\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.075368 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.075322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgvbz\" (UniqueName: \"kubernetes.io/projected/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-kube-api-access-pgvbz\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.075368 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.075351    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-dshm\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.075552 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.075390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tls-certs\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.176690 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.176658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-dshm\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.176853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.176711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tls-certs\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.176853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.176737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-model-cache\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.176853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.176769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-home\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.176853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.176804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tmp-dir\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.176853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.176820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgvbz\" (UniqueName: \"kubernetes.io/projected/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-kube-api-access-pgvbz\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.177196 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.177157    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-model-cache\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.177288 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.177212    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tmp-dir\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.177288 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.177253    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-home\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.179142 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.179122    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-dshm\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.179417 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.179395    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tls-certs\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.185210 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.185188    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgvbz\" (UniqueName: \"kubernetes.io/projected/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-kube-api-access-pgvbz\") pod \"rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") " pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.318404 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.318331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:48.454698 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:48.454673    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"]
Jul 07 00:25:48.457089 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:25:48.457061    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce29dcbd_f120_49ed_88f2_c74cb0108bbb.slice/crio-f520f7a125d527f93dea0bdddc238836c9ae8063ee1ad9ace04ff77b4b7fdb37 WatchSource:0}: Error finding container f520f7a125d527f93dea0bdddc238836c9ae8063ee1ad9ace04ff77b4b7fdb37: Status 404 returned error can't find the container with id f520f7a125d527f93dea0bdddc238836c9ae8063ee1ad9ace04ff77b4b7fdb37
Jul 07 00:25:49.428226 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:49.428190    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6" event={"ID":"ce29dcbd-f120-49ed-88f2-c74cb0108bbb","Type":"ContainerStarted","Data":"191a99892024242b7eab3eb913a7d0f8a49f9ec4ed8826a78d56f4f3df090357"}
Jul 07 00:25:49.428226 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:49.428230    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6" event={"ID":"ce29dcbd-f120-49ed-88f2-c74cb0108bbb","Type":"ContainerStarted","Data":"f520f7a125d527f93dea0bdddc238836c9ae8063ee1ad9ace04ff77b4b7fdb37"}
Jul 07 00:25:49.446934 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:49.446885    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6" podStartSLOduration=2.446871085 podStartE2EDuration="2.446871085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:25:49.445519321 +0000 UTC m=+2250.510851298" watchObservedRunningTime="2026-07-07 00:25:49.446871085 +0000 UTC m=+2250.512203062"
Jul 07 00:25:58.319402 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:58.319369    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:58.319402 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:58.319411    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:58.331712 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:58.331690    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:25:58.480859 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:25:58.480831    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:26:26.579838 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:26.579806    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"]
Jul 07 00:26:27.519684 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:27.519644    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"]
Jul 07 00:26:29.219416 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.219383    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"]
Jul 07 00:26:29.219905 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.219671    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd" podUID="96871ce9-1eaa-47c0-84fe-3570a9430d89" containerName="maas-api" containerID="cri-o://629fb4ee2e1c266a1100c787e4dfce28d7009ed0a1876e4582f13a9bc7735653" gracePeriod=30
Jul 07 00:26:29.617298 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.617262    2564 generic.go:356] "Generic (PLEG): container finished" podID="96871ce9-1eaa-47c0-84fe-3570a9430d89" containerID="629fb4ee2e1c266a1100c787e4dfce28d7009ed0a1876e4582f13a9bc7735653" exitCode=0
Jul 07 00:26:29.617510 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.617328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd" event={"ID":"96871ce9-1eaa-47c0-84fe-3570a9430d89","Type":"ContainerDied","Data":"629fb4ee2e1c266a1100c787e4dfce28d7009ed0a1876e4582f13a9bc7735653"}
Jul 07 00:26:29.668199 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.668175    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:26:29.740267 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.740228    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls\") pod \"96871ce9-1eaa-47c0-84fe-3570a9430d89\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") "
Jul 07 00:26:29.740496 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.740277    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96871ce9-1eaa-47c0-84fe-3570a9430d89-openshift-service-ca\") pod \"96871ce9-1eaa-47c0-84fe-3570a9430d89\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") "
Jul 07 00:26:29.740496 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.740386    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh8pg\" (UniqueName: \"kubernetes.io/projected/96871ce9-1eaa-47c0-84fe-3570a9430d89-kube-api-access-dh8pg\") pod \"96871ce9-1eaa-47c0-84fe-3570a9430d89\" (UID: \"96871ce9-1eaa-47c0-84fe-3570a9430d89\") "
Jul 07 00:26:29.740718 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.740691    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96871ce9-1eaa-47c0-84fe-3570a9430d89-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96871ce9-1eaa-47c0-84fe-3570a9430d89" (UID: "96871ce9-1eaa-47c0-84fe-3570a9430d89"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:26:29.742607 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.742581    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96871ce9-1eaa-47c0-84fe-3570a9430d89" (UID: "96871ce9-1eaa-47c0-84fe-3570a9430d89"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:26:29.742844 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.742823    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96871ce9-1eaa-47c0-84fe-3570a9430d89-kube-api-access-dh8pg" (OuterVolumeSpecName: "kube-api-access-dh8pg") pod "96871ce9-1eaa-47c0-84fe-3570a9430d89" (UID: "96871ce9-1eaa-47c0-84fe-3570a9430d89"). InnerVolumeSpecName "kube-api-access-dh8pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:26:29.842180 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.842146    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96871ce9-1eaa-47c0-84fe-3570a9430d89-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:29.842180 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.842175    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh8pg\" (UniqueName: \"kubernetes.io/projected/96871ce9-1eaa-47c0-84fe-3570a9430d89-kube-api-access-dh8pg\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:29.842180 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:29.842184    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96871ce9-1eaa-47c0-84fe-3570a9430d89-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:30.623187 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:30.623154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd" event={"ID":"96871ce9-1eaa-47c0-84fe-3570a9430d89","Type":"ContainerDied","Data":"795d552690c86bb51621de64d83633298de0a860b5484b04246f88aea3630463"}
Jul 07 00:26:30.623667 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:30.623189    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"
Jul 07 00:26:30.623667 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:30.623208    2564 scope.go:117] "RemoveContainer" containerID="629fb4ee2e1c266a1100c787e4dfce28d7009ed0a1876e4582f13a9bc7735653"
Jul 07 00:26:30.646178 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:30.646140    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"]
Jul 07 00:26:30.647935 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:30.647914    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b60cd9-598f7768b6-wrnjd"]
Jul 07 00:26:31.390749 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:31.390710    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96871ce9-1eaa-47c0-84fe-3570a9430d89" path="/var/lib/kubelet/pods/96871ce9-1eaa-47c0-84fe-3570a9430d89/volumes"
Jul 07 00:26:41.599960 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.599906    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9" podUID="29d48968-b00c-4038-b0b9-f2afb99403b7" containerName="main" containerID="cri-o://5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c" gracePeriod=45
Jul 07 00:26:41.858463 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.858391    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:26:41.960797 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.960762    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-dshm\") pod \"29d48968-b00c-4038-b0b9-f2afb99403b7\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") "
Jul 07 00:26:41.960797 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.960803    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-home\") pod \"29d48968-b00c-4038-b0b9-f2afb99403b7\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") "
Jul 07 00:26:41.961055 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.960841    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d48968-b00c-4038-b0b9-f2afb99403b7-tls-certs\") pod \"29d48968-b00c-4038-b0b9-f2afb99403b7\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") "
Jul 07 00:26:41.961055 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.960914    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-tmp-dir\") pod \"29d48968-b00c-4038-b0b9-f2afb99403b7\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") "
Jul 07 00:26:41.961055 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.960959    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-model-cache\") pod \"29d48968-b00c-4038-b0b9-f2afb99403b7\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") "
Jul 07 00:26:41.961055 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.960994    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tdpz\" (UniqueName: \"kubernetes.io/projected/29d48968-b00c-4038-b0b9-f2afb99403b7-kube-api-access-2tdpz\") pod \"29d48968-b00c-4038-b0b9-f2afb99403b7\" (UID: \"29d48968-b00c-4038-b0b9-f2afb99403b7\") "
Jul 07 00:26:41.961264 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.961120    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-home" (OuterVolumeSpecName: "home") pod "29d48968-b00c-4038-b0b9-f2afb99403b7" (UID: "29d48968-b00c-4038-b0b9-f2afb99403b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:41.961264 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.961192    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "29d48968-b00c-4038-b0b9-f2afb99403b7" (UID: "29d48968-b00c-4038-b0b9-f2afb99403b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:41.961264 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.961204    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29d48968-b00c-4038-b0b9-f2afb99403b7" (UID: "29d48968-b00c-4038-b0b9-f2afb99403b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:41.961414 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.961317    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:41.961414 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.961329    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:41.961414 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.961338    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:41.963103 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.963079    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-dshm" (OuterVolumeSpecName: "dshm") pod "29d48968-b00c-4038-b0b9-f2afb99403b7" (UID: "29d48968-b00c-4038-b0b9-f2afb99403b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:41.963222 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.963114    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29d48968-b00c-4038-b0b9-f2afb99403b7-kube-api-access-2tdpz" (OuterVolumeSpecName: "kube-api-access-2tdpz") pod "29d48968-b00c-4038-b0b9-f2afb99403b7" (UID: "29d48968-b00c-4038-b0b9-f2afb99403b7"). InnerVolumeSpecName "kube-api-access-2tdpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:26:41.963222 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:41.963165    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29d48968-b00c-4038-b0b9-f2afb99403b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29d48968-b00c-4038-b0b9-f2afb99403b7" (UID: "29d48968-b00c-4038-b0b9-f2afb99403b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:26:42.061877 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.061839    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tdpz\" (UniqueName: \"kubernetes.io/projected/29d48968-b00c-4038-b0b9-f2afb99403b7-kube-api-access-2tdpz\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.061877 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.061868    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d48968-b00c-4038-b0b9-f2afb99403b7-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.061877 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.061879    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d48968-b00c-4038-b0b9-f2afb99403b7-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.542864 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.542822    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6" podUID="ce29dcbd-f120-49ed-88f2-c74cb0108bbb" containerName="main" containerID="cri-o://191a99892024242b7eab3eb913a7d0f8a49f9ec4ed8826a78d56f4f3df090357" gracePeriod=45
Jul 07 00:26:42.682354 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.682319    2564 generic.go:356] "Generic (PLEG): container finished" podID="29d48968-b00c-4038-b0b9-f2afb99403b7" containerID="5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c" exitCode=0
Jul 07 00:26:42.682855 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.682365    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"
Jul 07 00:26:42.682855 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.682412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9" event={"ID":"29d48968-b00c-4038-b0b9-f2afb99403b7","Type":"ContainerDied","Data":"5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c"}
Jul 07 00:26:42.682855 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.682478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9" event={"ID":"29d48968-b00c-4038-b0b9-f2afb99403b7","Type":"ContainerDied","Data":"a32c086e56ea21aeee809ed97de6b75d184a5f729da56b52248720dfa1b22b69"}
Jul 07 00:26:42.682855 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.682505    2564 scope.go:117] "RemoveContainer" containerID="5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c"
Jul 07 00:26:42.684234 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.684212    2564 generic.go:356] "Generic (PLEG): container finished" podID="ce29dcbd-f120-49ed-88f2-c74cb0108bbb" containerID="191a99892024242b7eab3eb913a7d0f8a49f9ec4ed8826a78d56f4f3df090357" exitCode=0
Jul 07 00:26:42.684348 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.684280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6" event={"ID":"ce29dcbd-f120-49ed-88f2-c74cb0108bbb","Type":"ContainerDied","Data":"191a99892024242b7eab3eb913a7d0f8a49f9ec4ed8826a78d56f4f3df090357"}
Jul 07 00:26:42.693505 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.693480    2564 scope.go:117] "RemoveContainer" containerID="5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c"
Jul 07 00:26:42.693889 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:26:42.693856    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c\": container with ID starting with 5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c not found: ID does not exist" containerID="5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c"
Jul 07 00:26:42.694115 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.693900    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c"} err="failed to get container status \"5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c\": rpc error: code = NotFound desc = could not find container \"5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c\": container with ID starting with 5d5ee24caf0455459222fd35e23d0ac254fdb289a7ccf6ed1ed9c1dd21ada53c not found: ID does not exist"
Jul 07 00:26:42.707285 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.706836    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"]
Jul 07 00:26:42.712091 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.712062    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b60cd9/rate-test-model-b60cd9-kserve-57ff5894b7-tjkf9"]
Jul 07 00:26:42.803757 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.803690    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:26:42.869375 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869343    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgvbz\" (UniqueName: \"kubernetes.io/projected/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-kube-api-access-pgvbz\") pod \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") "
Jul 07 00:26:42.869375 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869385    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tmp-dir\") pod \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") "
Jul 07 00:26:42.869642 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-model-cache\") pod \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") "
Jul 07 00:26:42.869642 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869470    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-home\") pod \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") "
Jul 07 00:26:42.869642 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869496    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-dshm\") pod \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") "
Jul 07 00:26:42.869642 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869523    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tls-certs\") pod \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\" (UID: \"ce29dcbd-f120-49ed-88f2-c74cb0108bbb\") "
Jul 07 00:26:42.869841 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869711    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce29dcbd-f120-49ed-88f2-c74cb0108bbb" (UID: "ce29dcbd-f120-49ed-88f2-c74cb0108bbb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:42.869841 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869771    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-home" (OuterVolumeSpecName: "home") pod "ce29dcbd-f120-49ed-88f2-c74cb0108bbb" (UID: "ce29dcbd-f120-49ed-88f2-c74cb0108bbb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:42.869841 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.869793    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce29dcbd-f120-49ed-88f2-c74cb0108bbb" (UID: "ce29dcbd-f120-49ed-88f2-c74cb0108bbb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:42.871714 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.871685    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-dshm" (OuterVolumeSpecName: "dshm") pod "ce29dcbd-f120-49ed-88f2-c74cb0108bbb" (UID: "ce29dcbd-f120-49ed-88f2-c74cb0108bbb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:42.871836 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.871735    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce29dcbd-f120-49ed-88f2-c74cb0108bbb" (UID: "ce29dcbd-f120-49ed-88f2-c74cb0108bbb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:26:42.871836 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.871737    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-kube-api-access-pgvbz" (OuterVolumeSpecName: "kube-api-access-pgvbz") pod "ce29dcbd-f120-49ed-88f2-c74cb0108bbb" (UID: "ce29dcbd-f120-49ed-88f2-c74cb0108bbb"). InnerVolumeSpecName "kube-api-access-pgvbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:26:42.970255 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.970217    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-home\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.970255 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.970248    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-dshm\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.970255 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.970257    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tls-certs\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.970531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.970269    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgvbz\" (UniqueName: \"kubernetes.io/projected/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-kube-api-access-pgvbz\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.970531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.970306    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-tmp-dir\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:42.970531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:42.970315    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce29dcbd-f120-49ed-88f2-c74cb0108bbb-model-cache\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:43.390994 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:43.390960    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29d48968-b00c-4038-b0b9-f2afb99403b7" path="/var/lib/kubelet/pods/29d48968-b00c-4038-b0b9-f2afb99403b7/volumes"
Jul 07 00:26:43.690713 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:43.690631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6" event={"ID":"ce29dcbd-f120-49ed-88f2-c74cb0108bbb","Type":"ContainerDied","Data":"f520f7a125d527f93dea0bdddc238836c9ae8063ee1ad9ace04ff77b4b7fdb37"}
Jul 07 00:26:43.690713 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:43.690647    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"
Jul 07 00:26:43.691189 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:43.690676    2564 scope.go:117] "RemoveContainer" containerID="191a99892024242b7eab3eb913a7d0f8a49f9ec4ed8826a78d56f4f3df090357"
Jul 07 00:26:43.707676 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:43.707651    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"]
Jul 07 00:26:43.711403 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:43.711380    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6e67e/rate-test-model-c6e67e-kserve-7c54458f6c-xrkp6"]
Jul 07 00:26:45.390585 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:26:45.390550    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce29dcbd-f120-49ed-88f2-c74cb0108bbb" path="/var/lib/kubelet/pods/ce29dcbd-f120-49ed-88f2-c74cb0108bbb/volumes"
Jul 07 00:27:12.185075 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.184979    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"]
Jul 07 00:27:12.185731 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.185214    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5" podUID="0066ef9e-45e7-4199-b549-2c6996f8b5bb" containerName="maas-api" containerID="cri-o://f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f" gracePeriod=30
Jul 07 00:27:12.631341 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.631313    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:27:12.741189 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.741147    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bz5n\" (UniqueName: \"kubernetes.io/projected/0066ef9e-45e7-4199-b549-2c6996f8b5bb-kube-api-access-4bz5n\") pod \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") "
Jul 07 00:27:12.741411 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.741257    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls\") pod \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") "
Jul 07 00:27:12.741411 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.741348    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ef9e-45e7-4199-b549-2c6996f8b5bb-openshift-service-ca\") pod \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\" (UID: \"0066ef9e-45e7-4199-b549-2c6996f8b5bb\") "
Jul 07 00:27:12.741731 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.741706    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0066ef9e-45e7-4199-b549-2c6996f8b5bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0066ef9e-45e7-4199-b549-2c6996f8b5bb" (UID: "0066ef9e-45e7-4199-b549-2c6996f8b5bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:27:12.743554 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.743525    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0066ef9e-45e7-4199-b549-2c6996f8b5bb-kube-api-access-4bz5n" (OuterVolumeSpecName: "kube-api-access-4bz5n") pod "0066ef9e-45e7-4199-b549-2c6996f8b5bb" (UID: "0066ef9e-45e7-4199-b549-2c6996f8b5bb"). InnerVolumeSpecName "kube-api-access-4bz5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:27:12.743677 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.743578    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0066ef9e-45e7-4199-b549-2c6996f8b5bb" (UID: "0066ef9e-45e7-4199-b549-2c6996f8b5bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:27:12.823439 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.823398    2564 generic.go:356] "Generic (PLEG): container finished" podID="0066ef9e-45e7-4199-b549-2c6996f8b5bb" containerID="f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f" exitCode=0
Jul 07 00:27:12.823625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.823473    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"
Jul 07 00:27:12.823625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.823487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5" event={"ID":"0066ef9e-45e7-4199-b549-2c6996f8b5bb","Type":"ContainerDied","Data":"f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f"}
Jul 07 00:27:12.823625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.823538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5" event={"ID":"0066ef9e-45e7-4199-b549-2c6996f8b5bb","Type":"ContainerDied","Data":"eae18a2999acd239d9c6f388b3b9e16ce4ebe8b3309612568b0308936157edb9"}
Jul 07 00:27:12.823625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.823569    2564 scope.go:117] "RemoveContainer" containerID="f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f"
Jul 07 00:27:12.832572 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.832551    2564 scope.go:117] "RemoveContainer" containerID="f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f"
Jul 07 00:27:12.832836 ip-10-0-130-92 kubenswrapper[2564]: E0707 00:27:12.832819    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f\": container with ID starting with f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f not found: ID does not exist" containerID="f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f"
Jul 07 00:27:12.832882 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.832844    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f"} err="failed to get container status \"f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f\": rpc error: code = NotFound desc = could not find container \"f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f\": container with ID starting with f1f0a1f1cd76a8b5981609cf000ba734f86a970ff92584617719e8af314d7b0f not found: ID does not exist"
Jul 07 00:27:12.842304 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.842279    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0066ef9e-45e7-4199-b549-2c6996f8b5bb-maas-api-tls\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:27:12.842304 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.842303    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ef9e-45e7-4199-b549-2c6996f8b5bb-openshift-service-ca\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:27:12.842470 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.842313    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bz5n\" (UniqueName: \"kubernetes.io/projected/0066ef9e-45e7-4199-b549-2c6996f8b5bb-kube-api-access-4bz5n\") on node \"ip-10-0-130-92.ec2.internal\" DevicePath \"\""
Jul 07 00:27:12.846531 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.846504    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"]
Jul 07 00:27:12.850792 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:12.850769    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c6e67e-77c6c645dc-mxlx5"]
Jul 07 00:27:13.391776 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:13.391741    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0066ef9e-45e7-4199-b549-2c6996f8b5bb" path="/var/lib/kubelet/pods/0066ef9e-45e7-4199-b549-2c6996f8b5bb/volumes"
Jul 07 00:27:23.651891 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:23.651834    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7777c48764-pj5gk_1965738a-61fc-4bab-b90d-a5d34a6fe670/maas-api/0.log"
Jul 07 00:27:23.982515 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:23.982485    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67c88b8ddf-tf74m_83df5ec1-4d14-4642-9d27-74e3ae5f1f92/manager/0.log"
Jul 07 00:27:24.324579 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:24.324501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-gj6kg_ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f/copy-manifests/0.log"
Jul 07 00:27:24.332874 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:24.332850    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-gj6kg_ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f/manager/0.log"
Jul 07 00:27:24.713675 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:24.713651    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-c297r_e212522e-8b82-4d91-811f-387210396974/postgres/0.log"
Jul 07 00:27:26.587377 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:26.587347    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-4wtbm_7899a232-f64b-488e-aae5-794960999bb4/manager/0.log"
Jul 07 00:27:27.396530 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:27.396501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8bzw4_a778d1f2-e282-45bb-a9f9-26009fda6af0/kube-auth-proxy/0.log"
Jul 07 00:27:27.735625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:27.735592    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7jv49_d0812424-e05f-4565-bfcc-ee3dd83e1bf3/payload-pre-processing/0.log"
Jul 07 00:27:27.860939 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:27.860911    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lnvtg_feb71c59-3bbe-427b-b893-fe5312ca8497/payload-processing/0.log"
Jul 07 00:27:28.309599 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:28.309568    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rtmnb_cd895005-8e94-49b1-ad66-d0015e051db8/main/0.log"
Jul 07 00:27:28.850082 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:28.850045    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-2hk8z_875b2e5e-9377-40cf-af33-b38ee9777b6d/main/0.log"
Jul 07 00:27:35.963585 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:35.963556    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s4bnc_9c4c5715-f342-45cd-a938-c93cf23382b5/global-pull-secret-syncer/0.log"
Jul 07 00:27:36.124536 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:36.124499    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gkjd4_90ba0ca7-eb97-4dbd-acff-67537d724463/konnectivity-agent/0.log"
Jul 07 00:27:36.151134 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:36.151075    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-92.ec2.internal_9f4db311b901a006fd4141a884f57806/haproxy/0.log"
Jul 07 00:27:41.684789 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:41.684702    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-4wtbm_7899a232-f64b-488e-aae5-794960999bb4/manager/0.log"
Jul 07 00:27:43.534917 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:43.534878    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-f6t6n_7bfa3930-3794-45ea-94ba-3b019691282f/kube-state-metrics/0.log"
Jul 07 00:27:43.559706 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:43.559664    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-f6t6n_7bfa3930-3794-45ea-94ba-3b019691282f/kube-rbac-proxy-main/0.log"
Jul 07 00:27:43.586148 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:43.586117    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-f6t6n_7bfa3930-3794-45ea-94ba-3b019691282f/kube-rbac-proxy-self/0.log"
Jul 07 00:27:43.619502 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:43.619475    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-77b456fbdb-t5jlj_ee01e8f0-4a6c-4013-951c-f89cffa559dd/metrics-server/0.log"
Jul 07 00:27:43.865844 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:43.865734    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fsmg9_cc8f0e2a-fd77-49b1-b8e3-3382a6671f62/node-exporter/0.log"
Jul 07 00:27:43.887485 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:43.887459    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fsmg9_cc8f0e2a-fd77-49b1-b8e3-3382a6671f62/kube-rbac-proxy/0.log"
Jul 07 00:27:43.911610 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:43.911580    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fsmg9_cc8f0e2a-fd77-49b1-b8e3-3382a6671f62/init-textfile/0.log"
Jul 07 00:27:44.038109 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.038075    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0908b5df-02f9-477d-a376-52bca100223b/prometheus/0.log"
Jul 07 00:27:44.062529 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.062503    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0908b5df-02f9-477d-a376-52bca100223b/config-reloader/0.log"
Jul 07 00:27:44.085224 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.085194    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0908b5df-02f9-477d-a376-52bca100223b/thanos-sidecar/0.log"
Jul 07 00:27:44.109586 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.109558    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0908b5df-02f9-477d-a376-52bca100223b/kube-rbac-proxy-web/0.log"
Jul 07 00:27:44.131877 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.131789    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0908b5df-02f9-477d-a376-52bca100223b/kube-rbac-proxy/0.log"
Jul 07 00:27:44.154133 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.154105    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0908b5df-02f9-477d-a376-52bca100223b/kube-rbac-proxy-thanos/0.log"
Jul 07 00:27:44.179478 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.179447    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0908b5df-02f9-477d-a376-52bca100223b/init-config-reloader/0.log"
Jul 07 00:27:44.211471 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.211441    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-m82tt_48f76fce-bac3-49e8-9079-86d572fe6cd2/prometheus-operator/0.log"
Jul 07 00:27:44.228743 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:44.228718    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-m82tt_48f76fce-bac3-49e8-9079-86d572fe6cd2/kube-rbac-proxy/0.log"
Jul 07 00:27:45.922853 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.922813    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"]
Jul 07 00:27:45.923347 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923301    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0066ef9e-45e7-4199-b549-2c6996f8b5bb" containerName="maas-api"
Jul 07 00:27:45.923347 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923321    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0066ef9e-45e7-4199-b549-2c6996f8b5bb" containerName="maas-api"
Jul 07 00:27:45.923347 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923337    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96871ce9-1eaa-47c0-84fe-3570a9430d89" containerName="maas-api"
Jul 07 00:27:45.923347 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923344    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96871ce9-1eaa-47c0-84fe-3570a9430d89" containerName="maas-api"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923357    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d48968-b00c-4038-b0b9-f2afb99403b7" containerName="main"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923366    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d48968-b00c-4038-b0b9-f2afb99403b7" containerName="main"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923388    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce29dcbd-f120-49ed-88f2-c74cb0108bbb" containerName="main"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923395    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce29dcbd-f120-49ed-88f2-c74cb0108bbb" containerName="main"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923504    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce29dcbd-f120-49ed-88f2-c74cb0108bbb" containerName="main"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923518    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96871ce9-1eaa-47c0-84fe-3570a9430d89" containerName="maas-api"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923526    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29d48968-b00c-4038-b0b9-f2afb99403b7" containerName="main"
Jul 07 00:27:45.923625 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.923538    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0066ef9e-45e7-4199-b549-2c6996f8b5bb" containerName="maas-api"
Jul 07 00:27:45.926751 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.926730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:45.929444 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.929405    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s7kft\"/\"default-dockercfg-grvfn\""
Jul 07 00:27:45.929537 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.929484    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s7kft\"/\"openshift-service-ca.crt\""
Jul 07 00:27:45.929537 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.929506    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s7kft\"/\"kube-root-ca.crt\""
Jul 07 00:27:45.934456 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:45.934415    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"]
Jul 07 00:27:46.036850 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.036815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-podres\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.036850 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.036858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-proc\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.037080 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.036886    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-sys\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.037080 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.036945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-lib-modules\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.037080 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.036996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvlgt\" (UniqueName: \"kubernetes.io/projected/e32ae946-3123-44f3-992f-de0e3a95aa56-kube-api-access-xvlgt\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137703 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-podres\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137703 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-proc\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137924 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137742    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-sys\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137924 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-lib-modules\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137924 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvlgt\" (UniqueName: \"kubernetes.io/projected/e32ae946-3123-44f3-992f-de0e3a95aa56-kube-api-access-xvlgt\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137924 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137853    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-sys\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137924 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137853    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-podres\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137924 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137893    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-proc\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.137924 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.137906    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e32ae946-3123-44f3-992f-de0e3a95aa56-lib-modules\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.146159 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.146136    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvlgt\" (UniqueName: \"kubernetes.io/projected/e32ae946-3123-44f3-992f-de0e3a95aa56-kube-api-access-xvlgt\") pod \"perf-node-gather-daemonset-nbt8r\" (UID: \"e32ae946-3123-44f3-992f-de0e3a95aa56\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.238174 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.238140    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:46.367887 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.367863    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"]
Jul 07 00:27:46.370065 ip-10-0-130-92 kubenswrapper[2564]: W0707 00:27:46.370031    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode32ae946_3123_44f3_992f_de0e3a95aa56.slice/crio-cee8542d52c0c2cfd92b4f803f01375372d87dd043a74ebc22a14a6451c6b9d5 WatchSource:0}: Error finding container cee8542d52c0c2cfd92b4f803f01375372d87dd043a74ebc22a14a6451c6b9d5: Status 404 returned error can't find the container with id cee8542d52c0c2cfd92b4f803f01375372d87dd043a74ebc22a14a6451c6b9d5
Jul 07 00:27:46.965835 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.965802    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5ddc7dcb8f-mgh8k_57af95ba-10a3-46b4-9c56-c9cfd4c7b050/console/0.log"
Jul 07 00:27:46.987976 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.987939    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r" event={"ID":"e32ae946-3123-44f3-992f-de0e3a95aa56","Type":"ContainerStarted","Data":"eae8d1d51111265fae8d11163f950801082eb44ab17772a9d3f32ff1db39c589"}
Jul 07 00:27:46.988135 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.988033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r" event={"ID":"e32ae946-3123-44f3-992f-de0e3a95aa56","Type":"ContainerStarted","Data":"cee8542d52c0c2cfd92b4f803f01375372d87dd043a74ebc22a14a6451c6b9d5"}
Jul 07 00:27:46.988135 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:46.988062    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:47.005967 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:47.005921    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r" podStartSLOduration=2.005907916 podStartE2EDuration="2.005907916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:27:47.003923063 +0000 UTC m=+2368.069255039" watchObservedRunningTime="2026-07-07 00:27:47.005907916 +0000 UTC m=+2368.071239899"
Jul 07 00:27:48.734171 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:48.734124    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nl57w_a8d25186-1b3b-4fcd-a596-6592eb4507ab/dns/0.log"
Jul 07 00:27:48.756525 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:48.756499    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nl57w_a8d25186-1b3b-4fcd-a596-6592eb4507ab/kube-rbac-proxy/0.log"
Jul 07 00:27:48.804202 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:48.804172    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sgk8d_d1048959-28a5-4e60-922b-4c6af41cafc7/dns-node-resolver/0.log"
Jul 07 00:27:49.399061 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:49.399031    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cqmp5_7e740413-5fb8-490c-8254-dfc1d73a84ac/node-ca/0.log"
Jul 07 00:27:50.573709 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:50.573672    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8bzw4_a778d1f2-e282-45bb-a9f9-26009fda6af0/kube-auth-proxy/0.log"
Jul 07 00:27:50.722795 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:50.722758    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7jv49_d0812424-e05f-4565-bfcc-ee3dd83e1bf3/payload-pre-processing/0.log"
Jul 07 00:27:50.798293 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:50.798261    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lnvtg_feb71c59-3bbe-427b-b893-fe5312ca8497/payload-processing/0.log"
Jul 07 00:27:51.511298 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:51.511272    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7g28g_58128f39-84e1-43a5-bcdd-91dbdbb82ee2/serve-healthcheck-canary/0.log"
Jul 07 00:27:52.187267 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:52.187230    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s8jnd_4d6f7503-aaab-4525-a863-8a3daee84210/kube-rbac-proxy/0.log"
Jul 07 00:27:52.211372 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:52.211342    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s8jnd_4d6f7503-aaab-4525-a863-8a3daee84210/exporter/0.log"
Jul 07 00:27:52.235480 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:52.235456    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s8jnd_4d6f7503-aaab-4525-a863-8a3daee84210/extractor/0.log"
Jul 07 00:27:53.002772 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:53.002746    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-nbt8r"
Jul 07 00:27:54.862607 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:54.862579    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7777c48764-pj5gk_1965738a-61fc-4bab-b90d-a5d34a6fe670/maas-api/0.log"
Jul 07 00:27:55.074342 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:55.074309    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67c88b8ddf-tf74m_83df5ec1-4d14-4642-9d27-74e3ae5f1f92/manager/0.log"
Jul 07 00:27:55.237363 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:55.237177    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-gj6kg_ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f/manager/0.log"
Jul 07 00:27:55.264275 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:55.264252    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-gj6kg_ada9f195-fdaa-4c3d-82bb-6e9d2bf1300f/copy-manifests/0.log"
Jul 07 00:27:55.458643 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:55.458604    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-c297r_e212522e-8b82-4d91-811f-387210396974/postgres/0.log"
Jul 07 00:27:56.622595 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:56.622571    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8bzw4_a778d1f2-e282-45bb-a9f9-26009fda6af0/kube-auth-proxy/0.log"
Jul 07 00:27:56.734038 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:56.733996    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7jv49_d0812424-e05f-4565-bfcc-ee3dd83e1bf3/payload-pre-processing/0.log"
Jul 07 00:27:56.798007 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:56.797974    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lnvtg_feb71c59-3bbe-427b-b893-fe5312ca8497/payload-processing/0.log"
Jul 07 00:27:56.884780 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:56.884697    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66459bdc77-bsgkb_d14a6c33-8f44-4f25-9a8f-0d3d71fb5898/manager/0.log"
Jul 07 00:27:58.163258 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:58.163228    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8bzw4_a778d1f2-e282-45bb-a9f9-26009fda6af0/kube-auth-proxy/0.log"
Jul 07 00:27:58.278960 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:58.278927    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7jv49_d0812424-e05f-4565-bfcc-ee3dd83e1bf3/payload-pre-processing/0.log"
Jul 07 00:27:58.350286 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:27:58.350220    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lnvtg_feb71c59-3bbe-427b-b893-fe5312ca8497/payload-processing/0.log"
Jul 07 00:28:03.184223 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.184192    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8nck4_8fc4d22f-e22b-4597-b7ff-572393b0154c/kube-multus/0.log"
Jul 07 00:28:03.215205 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.214921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-49rpw_0fd3b1e7-7b3d-436b-b950-23cda301c067/kube-multus-additional-cni-plugins/0.log"
Jul 07 00:28:03.243099 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.243071    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-49rpw_0fd3b1e7-7b3d-436b-b950-23cda301c067/egress-router-binary-copy/0.log"
Jul 07 00:28:03.267059 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.267029    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-49rpw_0fd3b1e7-7b3d-436b-b950-23cda301c067/cni-plugins/0.log"
Jul 07 00:28:03.289706 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.289676    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-49rpw_0fd3b1e7-7b3d-436b-b950-23cda301c067/bond-cni-plugin/0.log"
Jul 07 00:28:03.311764 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.311739    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-49rpw_0fd3b1e7-7b3d-436b-b950-23cda301c067/routeoverride-cni/0.log"
Jul 07 00:28:03.333724 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.333701    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-49rpw_0fd3b1e7-7b3d-436b-b950-23cda301c067/whereabouts-cni-bincopy/0.log"
Jul 07 00:28:03.357086 ip-10-0-130-92 kubenswrapper[2564]: I0707 00:28:03.357051    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-49rpw_0fd3b1e7-7b3d-436b-b950-23cda301c067/whereabouts-cni/0.log"